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Foreword 

The  importance  of  the  study  of  the  solution  of  systems  of  Unear  equations  and  the 
determination  of  eigenvalues  has  long  been  recognized.  Many  problems  find  their 
most  direct  and  natural  formulations  as  s3-stems  of  linear  equations.  Then,  too,  often 
a  system  of  linear  equations  fiu-nishes  the  most  convenient  tool  for  an  approximate 
analysis  of  a  contmuous  mathematical  model;  classical  cases  are  the  approximate  solu- 
tion of  differential  equations  by  difference  equations  and  the  analogous  discrete  approach 
to  integral  equations.  Similarly,  the  determination  of  eigenvalues  of  continuous  or 
discrete  vibrating  systems  is  a  fundamental  problem  in  both  classical  and  modern 
theoretical  physics. 

With  the  advent  of  the  high  speed  automatic  digital  computmg  machines  there 
came  a  compelling  reason  to  take  another  look  at  these  problems,  and  to  study  them  in 
the  light  of  the  equipment  now  being  made  available.  This  reexamination  has  been 
going  on  for  some  ten  3'ears  in  various  centers  and  a  number  of  interesting  new  results 
have  been  announced.  It  therefore  seemed  desirable  to  organize  a  Symposium  on 
these  topics,  where  active  workers  coidd  meet  and  exchange  ideas.  The  generous 
support  of  the  Office  of  Naval  Research  on  the  occasion  of  the  National  Bureau  of 
Standards  Semicentennial  celebration  made  a  comprehensive  gathering  possible.  It 
took  place  at  the  National  Bureau  of  Standards  Institute  for  Numerical  Analysis  at 
Los  Angeles  on  August  23-25,  1951. 

The  choice  of  location  was  particularly  apnropriate,  for  the  mathematicians  of  the 
NBS  Institute  for  Numerical  Analysis  have  been  devoting  a  considerable  amount  of 
effort  to  the  topic  of  the  Symposium.  This  specialization  began  under  the  leadership 
of  Dr.  J.  Barkle}'  Rosser  in  1949-1950;  its  results  are  partially  reflected  in  the  reports 
presented  at  the  Symposium  by  members  of  the  National  Bureau  of  Standards'  Staff. 

The  teclmical  organization  of  the  meeting  was  mainlv  the  responsibility  of  Dr. 
Olga  Taussky-Todd,  who,  with  the  assistance  of  Dr.  L.  J.  Paige,  edited  the  present 
volume. 

The  Symposium  consisted  of  invited  reports  from  various  mathematicians,  on 
selected  topics,  followed  by  a  round-table  discussion.  The  reports  on  simultaneous 
linear  equations  were  confined  to  finite  systems,  but  those  on  the  determination  of 
eigenvalues  dealt  with  both  the  discrete  and  continuous  cases,  and  included  reports 
specifically  on  the  determination  of  bounds  for  the  eigenvalues.  The  present  volume 
contains  the  majority  of  the  reports  which  were  presented  at  the  meeting. 

A.  V.  AsTiN,  Director, 
National  Bureau  of  Standard/^. 
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1.  Tentative  Ciassification  of  Methods  and  Bibliography 
on  Solving  Systems  of  Linear  Eqnations* 

By  George  E,  Forsythef 

I.  Introduction 

The  scope  of  this  article  is  indicated  in  the  introductions  to  sections  II,  IV,  and  especially  V.  In  the 
same  places  will  be  found  acknowledgments  to  some  of  the  persons  who  have  helped  build  the  bibliog- 
raphy. 

This  article  is  a  collection  of  working  notes,  and  not  a  polished  bibliography.  Because  there  is 
apparently  nothing  comparable  in  the  literature,  it  seems  better  to  publish  it  as  it  is  than  to  wait  for  a 
state  of  completion  which  might  never  arrive.  Although  many  of  the  references  in  section  V  have  been 
ch(>cked  in  the  course  of  the  preparation  of  the  manuscript,  time  has  not  permitted  a  systematic  editing 
and  search  for  errors.**  If  readers  will  send  in  lists  of  errors  and  omissions,  it  ma}^  later  be  possible  to 
issue  a  corrected  edition  of  this  article.  Please  address  the  compiler  at  the  National  Bureau  of  Standards, 
405  Hilgard  Avenue,  Los  Angeles  24,  Calif. 

II.   Basis  of  the  Tentative  Classification;  Symbols  Used  in  the  Outline 

We  are  here  dealing  with  methods  for  getting  the  solution  A~^b  of  a  system  of  linear  equations 
Ax=b,  where  A  is  a,  nonsingular  square  matrix  of  order  n  and  6  is  a  column  vector.  We  are  equally 
concerned  with  methods  for  obtaining  the  inverse  matrix  A~^;  where  such  methods  simply  parallel  those 
for  getting  A~^b,  they  will  not  ordinarily  be  mentioned.  Since  the  only  known  precedent  for  a  classifica- 
tion of  methods,  Jensen  1944,^  covers  but  a  fraction  of  the  methods  considered  here,  it  was  necessarv  to 

.  .  .       •  .  .  .     " 

devise  some  basis  for  the  outline  of  section  III.     The  basis  was  developed  after  conversations  with  Dr. 

Theodore  S.  Motzkin,  and  is  regarded  as  only  tentative. 

Consider,  for  the  moment,  an  iterative  process  for  getting  A~^b.  Starting  from  b,  A,  and  some  initial 
vector  ^^'",  a  set  of  arithmetic  operations  is  prescribed,  which,  if  (tarried  out  without  round-oft"  or  other 
error,  will  yield  a  new  vector  ^^^^:  ^'■^'>  =  Fi(b,A;^^'^'>).  Similarly,  ^^^'  is  obtained  by  another  set  of  opera- 
tions, carried  out  without  error.  In  general,  ^^*>  is  obtained  from  b,  A,  ^^*~^\  ^^*~^^  •  •  •,  ^^"^  by  applying 
a  set  of  exact  aritlunetic  operations  which  we  assume  to  be  uniquely  described  by  a  formula  of  type 

In  this  mann(>r  there  is  defined  a  theoretically  determined  sequence  of  vectors 

^''\e'\^"\   .   .   .,^'*>,   .   .   .. 

llowevei-,  round-oft"  errors  occur  in  almost  all  numerical  processes  (there  are  exceptions).  As  a 
cumulative  result  of  these  errors,  one  actually  obtains  a  vector  x^'^  instead  of  ^^'^  In  the  next  step,  one 
ordinarily  uses  b,  A,  and  y''^  in  the  algoritiun  designed  to  compute  ^<^*.  Suppose  that  tj<^>  would  be  the 
result  of  errorless  application  of  the  algorithm: 


•Tile  prcpiirulion  of  this  pupcr  was  sponsored  (in  pari)  by  Ihp  Ofllce  of  the  Air  Comptroller,  I'SA  V,  and  the  (JfFiee  of  Naval  Hesearch. 
fXalional  Bureau  of  Standards,  Los  AnRclcs,  Calif. 

••In  readlnR  the  R.illey  proofs,  the  non-bracketed  parts  of  the  entries  In  section  V  have  been  corrected  against  the  originnl  papers  or  their  reviews  in  MR, 
Zb,  Fs,  or  SA  (sec  section  V),  except  for  the  [icrhafis  15  to  20  titles  neither  contained  nor  reviewed  In  the  NBS  Institute  for  Nurnerleil  Analysis. 
'  Names  in  capitals  refer  to  the  bibliography  iii  section  V. 


where  x^°^  =  ^^^\  Because  of  round-off  errors,  one  really  obtains  not  r]^^\  but  another  uniquely  defined 
vector  x^^': 

Here  F2  is  the  function  F2,  altered  in  accord  with  each  of  the  round-off  errors  entering  the  calculation. 
In  this  manner  will  be  determined  a  theoretical  sequence  of  vectors 

^(0)(  =  ^(0))^^a)^^(2)^   .   .   .,  V*>,   .   .   ., 
and  a  practical  sequence 

2.(0)/-_fc(0)')   j.a)  2^(2)       _     _     _     y.(k) 

Each  vector  x'-"''  is  determined  from  its  precedessors  by  the  "rounded-off  function"  Fki 

x'''^  =  Fk{h,A;x'^-^\x'''-^\  .  .   .,a;(o')- 

Each  vector  77^*'  is  the  result  of  errorless  application  of  the  theoretical  function  Fk  to  the  "contaminated" 
sequence  a:® \  .  .  .,a;^*~^^: 

■r^^^^  =  Fk{h,A;x^^-^\x^^-^\  .  .  .,a;(«0- 

We  thus  have  three  sequences  in  mind:  (1)  the  sequence  {^'*^}  of  vectors  free  from  round-off  error 
in  all  generations  (it  is  these  theoretical  vectors  with  which  convergence  proofs  generally  deal),  (2)  the 
sequence  {x^*^}  of  vectors  which  the  computer  actually  obtains  while  making  the  prescribed  round-off 
errors,  and  (3)  the  sequence  {77'^^}  of  vectors  which  have  one  ''generation"  of  errorless  calculation,  based 
on  the  computed  vectors  x^^\  .  .  .,  x^'^~^\ 

We  have  supposed  heretofore  that  we  deal  with  an  iterative  process,  but  the  same  analysis  applies 
also  to  a  direct  process- — for  example,  elimination.  The  result  of  an  entire  elimination  solution  is  theo- 
retically the  vector  ^^^''=A~^h.  In  practice  one  makes  round-off  errors  and  obtains  x'-^^  instead.  Since 
h—Ax^'^^=^r^'^^7^0,  it  is  customary  to  solve  the  equation  .4(tj^^^— x^^^)=r'^^  for  the  correction  to  be  added 
to  x^^^  to  obtain  -q^^K  Here  r]^^^=A~'^h,  but  round-off  errors  cause  the  approximate  solution  a;^^^  to  be 
obtained  instead,  etc.  Thus  the  earlier  model  describes  also  the  "direct"  solution  of  the  system  Ax=b 
whenever  round-off  errors  are  made  in  the  actual  solution. 

The  present  classification  of  methods  is  based  on  two  trioJiotomies. 

First  trichotomy:  The  sequence  {r?^*'}  is  a  priori  bound  to  be  of  one  of  the  following  three  types: 

I.  All  77'*^  are  necessarily  equal  to  A~^b,  i.  c.,  all  are  exact. 

II.  Some  r?^*^  but  not  all,  are  necessarily  equal  to  A~^b. 

III.  No  T7^*^  is  equal  to  A~^b,  except  for  special  choices  of  r?'"^  b,  A. 

Typical  processes  of  these  three  types  are  I,  the  elimination  process  of  Gauss  1826;  II,  the  finite  iteration 
of  Lanczos  1951 ;  and  III,  the  infinite  iteration  of  Seidel  1874. 

Second  trichotomy:  The  sequence  [Fk]  of  functions  is  bound  a  priori  to  fall  into  one  of  the  following 
three  classes: 

a.  All  functions  Fk  have  in  fact  the  same  number  of  arguments  and  are  identical,  i.  e.,  there  exists 
an  integer,  r,  such  that 

(*)  Fk(b,A;x''^-'\.  .   .,x(o')^F(6,A;a;«*-l^.   .   .,x<*-^^)  (all  it). 

Such  processes  are  called  stationary. 

jS.  There  are  a  finite  number  (greater  than  one)  of  distinct  functions  Fk,  each  of  type  (*).  Such 
processes  are  called  partly  stationary. 

y.  There  are  an  infinite  number  of  distinct  functions  Fk.  Such  processes  are  called  nonstationary . 
Typical  processes  of  these  types  are  a,  (iterated)  ehmination;  /S,  iterated  elimination,  occasionally  inter- 
rupted by  an  acceleration  procedure;  7,  weighted  averages  of  the  successive  iterates  of  Seidel's  process, 
with  weights  equal  to  the  coeflScients  of  the  Chebyshev  polynomials  of  order  1,  2,  3,  .  .  .  .  (Di\dsions 
between  processes  of  types  /3  and  7  could  be  made  in  many  different  ways.) 

The  double  trichotomy  gives  us  a  priori  a  nine-way  classification  of  methods  for  gettmg  A~^b  (or 
indeed  of  estimating  any  quantity  by  a  numerical  process).     Three  of  the  possible  types,  1/3,  I7,  and 


II/3,  do  not  seem  to  occur  in  the  literature  on  getting  ^.1  '6,  but  the  other  six  form  a  convenient  structure 
on  which  to  hang  an  outline  of  methods  of  solvmg  linear  equations: 


a 

Stationary 

process 

0 

Partly 

stationary 

process 

y 

Nonstationary 

process 

I.  All  77 («  exact .-.     _       ..---.-          .   .   -_. 

la 
Ila 

11.  Some  i;^*>  exact 

III.  No  77<*>  exact--   . .   -   ._ 

UB 

Ilia                        Ills 

III7 

It  is  not  presumed  that  this  particular  nine-way  classification  is  necessarily  a  "good"  one;  it  merely 
provides  one  of  many  possible  frameworks  on  which  to  spread  out  the  known  solution  processes. 

The  outline  in  section  III  serves  two  purposes:  (1)  It  contains  a  tentative  classification  of  methods 
for  obtaming  A~^  or  A~^b;  (2)  it  provides  subject  headings  for  the  bibliography  of  section  V.  Purpose 
(1)  is  achieved  by  the  portion  of  the  outline  in  which  the  six  major  headings  la,  IIcc,  II/3,  IIIo:,  III/3, 
III7  are  subdivided.  In  these  the  methods  are  classified  according  to  the  nature  of  the  iteration  func- 
tions found  in  the  literature.  No  strenuous  effort  has  been  made  to  bring  all  closely  related  methods 
into  a  single  entry,  even  where  this  could  be  done  within  the  conventional  outline  structure,  for  it  is 
felt  that  purpose  (2)  is  better  served  without  too  great  a  condensation.  There  are  some  methods  that 
belong  at  two  distinct  points  of  the  outline  (c.  g.,  Scidel's  at  10c  and  at  13a  {1  &  5)). 

Purpose  (2)  requires  not  only  the  above  subdivisions,  but  also  the  heading  0  (surveys),  and  those 
under  IV  (miscellaneous  relevant  topics)  and  V  (contents  unlaiown). 

A  good  many  words  could  be  said  in  explanation  of  the  outline  and  its  notation,  but  it  is  hoped 
that  the  reader  will  be  able  to  make  it  out  well  enough.     Let  it  suffice  to  define  the  symbols  used: 
^  =(«,_,)  =nonsingular  square  matrix  of  order  n; 
6  =  column  vector; 
/(^4)=  characteristic  polynomial  of  A; 
/=unit  matrix; 
^~^= inverse  of  A; 
yl^=  transpose  of  A; 
"^4^0"  means  A  is  positive  definite; 

5^  process:  see  Aitken  1925,  1937b,  1950; 
x^'^=kth.  approximating  vector  to  A~^b; 

5=matrix  approximating  to  A  which  is  efi^ectively  inverted  in  processes  of  type  10; 
\x\R=(x'^Rx)^  =  \ength  oi  x  in.  R  metric; 

e<=unit  column  vector  with  1  for  ith  component  and  0  for  all  other  components; 
X^*'  is  kih  approximating  matrix  to  A~^; 

c?i=some  column  vector; 
Aa;<*>=a;<*+'>-j;<*>; 
P,  <?  =  nonsingiilar  square  matrices  of  order  n. 


III.   Tenlalive  Outline  of  Mrlliods 


0. 


la. 


Survey  of  methods. 

a.  Mainly  iterative  methods. 

b.  Mainly  direct  methods. 

c.  Contains  references  not  included  in  section  V. 
All  answers  theoretically  exact.     Stationary  process. 

1.     Solve  explicitly. 

a.  By  determinants. 

1.     Efficient  evaluation  of  determinants  (often  same  as  2a). 

b.  Otherwise. 


2.  Triangularize  A  and  solve. 

a.  By  elimination  (almost  same  as  3ai). 

1.  Inexact. 

A.     Efficient  arrangements. 

2.  Exact. 

A.     Controlled  magnitude  of  numbers. 

b.  By  extension  =  escalator  processes  (related  to  4a). 
1.     By  y4-ortliogonalization  of  unit  vectors. 

c.  By  generalized  Cholesky  method  (A  symmetrical  or  not). 
1.     Square-root  method  (symmetric^). 

d.  Any  of  above,  with  search  for  pivots. 

e.  Any  of  above,  with  "below-the-line"  arrangement. 

f.  Any  of  above,  using  Cracovians. 

3.  Orthogonalize  A  and  solve. 

a.  By  triangularization. 

1.     Elimination  (almost  same  as  2a). 

3.  Complete  elimination. 

3.     Above,  with  search  for  pivots. 

b.  By  orthogonalizing  rows  or  columns  of  A. 

4.  Partition  A  into  blocks,  and  carry  out  processes  2  or  3  on  the  blocks. 

a.  Smaller  matrices  inverted  by  a  direct  process. 

b.  Smaller  matrices  inverted  by  iterative  process. 

5.  Iterate  any  of  above. 

I  la.     Some  answers  theoretically  exact.     Stationary  jrrocess. 

6.  Methods  related  to  characteristic  polynomial/  of  some  matrix  H. 

a.  Calculate /(.4)  from  determinant  d{A  —  \I). 
1 .     Use  Newton's  identities. 

b.  Finite  iteration  to  get /(.A). 

1.  Matrix  iteration. 

2.  Vector  iteration. 

c.  Process  Qh2  applied  to  iterates  of  an  infinite  iteration,  to  getf(H). 

d.  Methods  related  to  successively  orthogonalizing  the  vectors  Ax,  A'x,  A:^x,  A^x, 

1 .  In  /  metric. 

2.  In  A-^  metric  (when  yl=A^>0). 
7.     Not  related  to  characteristic  polynomials. 

a.  Use  of  theory  of  congruences  in  n  dimensions. 

b.  Replacement  methods  (closely  related  to  2). 

1.  Simplex  method. 

2.  Replace  one  column  of  A  at  a  time. 

II|3.       Some  answers  theoretically  exact.     Partly  stationary  process. 

8.  Occasionally  interrupt  the  iteration  of  process  of  type  la  by  accelerating  with  .   . 

a.  ...  52  process  (2lhl). 

b.  ...  linear  combinations  of  prior  answers  (i.  e.,  apply  21a)). 

c.  ...  other  process. 

9.  Occasionally  interrupt  the  iterative  cycles  of  process  of  type  Ila  by  accelerating  . 

a.  ...  with  8^  process  (21hl). 

b.  ...  with  linear  combinations  of  prior  answers  (i.  e.,  apply  21ai). 

c.  ...  by  starting  over  from  current  residual  (for  process  6d). 

d.  ...  with  other  process. 

Ilia.     No  answer  theoretically  exact.     Stationary  jrrocess. 

10.     Use  a  linear  iteration  function :  x^''^^^  =x^''^ -^B-^h  —  Ax^^^). 

a.  General  process  of  first  degree. 

b.  5= diagonal  of  A  (Jacobi). 


f. 


5= lower  triangle  of  A  (Seidel).* 
B= scalar  matrix  (e.  g.,  Neumann  series). 
B  built  of  submatrix  blocks. 
1.     Blocks  inverted  by  type  I  process. 
3.     Blocks  inverted  b}^  type  II  process. 
Reflection  in  hyperplanes. 
g.     Linear  processes  of  second  or  higher  degree  (actually  21&1). 
11.     Use  a  nonlinear  iteration  function. 

a.  Use  a  least-squares  process  with  hcrmitian  matric  R  (here  Tr  —  ir(x)  determines  the  sub- 

space  in  which  one  minimizes  j^x— 6!^.). 

1.  ir=Ax—b  (one  dimension). 

2.  Tr= A'^(Ax—h)  (one  dimension). 

3.  IT  has  2  or  more  dimensions. 

4-     Systematically  undershoot  or  overshoot  point  of  minimum  \Ax—b\R. 

5.  R=I. 

6.  R=A-'  (when  A=A''yO). 

7.  R=^{AA'')-\ 

8.  Other  R. 

b.  Use  nonhermitian  metric  or  nonmetric  process. 


(choice  of 
one) 


1 .     Use  piecewise  linear  iteration  (relaxation  methods) :  For  some  i  maximizing 
A.     ...   \(Ax—b)i\,  change  .  .  . 
7).     .   .  .   \{Ax—b)i\l\aii\,  change  .  .  . 
.  .   \(Ax—b)i\/\aii\^,  change 


III^. 


C.  . 

D.  .  .  .   \[A^{Ax-b)U/{A'^A)uK  change  .  .  . 

E.  ...  something  else,  change  .  .  . 

F.  .  .  .  d  (unit  vector)  so  as  to  bring  .... 
C-.     .  .  .  A^Ci  so  as  to  bring  .  .  . 
H.     ...  something  else  so  as  to  bring  .  .  . 
/.     ...   {Ax—b)i  approximately  to  0. 
J.     ...  [A'''{Ax—b)]i  approximately  to  0. 
K.     ...  something  else  to  0. 
L.     With  any  scheme  above,  systematically  overshoot  or  undershoot  point  where 

residual  becomes  0  (over  and  undcrrelaxing). 
Use  piecewise  linear  iteration  (block  relaxation  methods):  For  some  two  or  more  i, 
change  two  or  more  .  .  . 

A.  .  .  .  Ci  so  as  to  minimize  something. 

B.  .  .  .  A^Bi  so  as  to  minimize  something. 
Get  A~^  with  polynomial  iteration  function: 

1.  Use  .Y<*+')=X<*)  (2/-^Z(*>). 

2.  Modify  ^ 

3.  Use  related  polynomials  of  Ihird  or  higher  oider. 
No  answer  theoretically  exact.     Partly  .stationary  process. 

12.     Occasionally  interrupt  the  iteration  of  process  of  type  Illa  by  iucclerating  widi   .   .   . 
a.     ...  g2  process  (21b/). 


(choice  of 
one) 

(choice  of 
one) 


2. 

(choice  of  J 
one)       [ 
c. 


b. 
c. 

d. 

e. 
f. 


step  of  type  1  la.^  (in  types  11  a). 

change  of  w. 

step  of  type  \lh2  Cm  types  llb/j. 

change  of  B  (in  type  10). 

linear  combination  of  prior  answers  (i.  e.,  a]jph'  21a/). 


•Aiidcd  ill  pr(Kjf:  This  Is  commonly  called  Scldel's  process.     Professor  Ostrowskl  has  pointed  out,  however,  that  Seidei,  1S74  advociled  only  the  process  llbl 
(C&  F  &  I),  while  the  process  10c  was  apparently  first  advocated  by  Nekbasov  1884  (see  Second  Supplement),  who  attribute  1  It  to  Seidel. 


209431— .5.3- 


13.  Use  m  iteration  functions  in  cyclic  order  (when  consolidated  in  cycles,  process  becomes  typ( 

Ilia.) 

a.  Use  least-squares  process  with  hermitian  metric  E,  minimizing    along    directions  di 

(?2,  •   •  •   ,  dji,  and  repeat. 
1.     c?j=g,  (unit  vectors). 

3.  Systematically  undershoot  or  overshoot  point  of  minimum  \Ax—b\R. 

4.  R=I. 

5.  R=A-'  (when  yl=^^>0,  13a  {1  &  5)  =  Seidel  process  10c). 

6.  R=(AA'')-'. 

7.  Other  R. 

b.  Use   least-squares   process  with   hermitian   metric  R,  minimizing  along  directions  di 

c?2,  .   .  .,  dn,  dn-\,  .  .  .,  d2,  aud  repeat. 

1.  di=ei  (unit  vectors),  R=A~^  (back-and-forth  Seidel  process). 

2.  di=ei,  general  R. 

c.  Use  m  linear  iteration  functions  of  type  10a  and  repeat. 
1.     Use  them  in  a  "back-and-forth"  order. 

d.  Use  m  nonlinear  iteration  functions. 

14.  Use  one  of  n  separate  processes  of  type  10a  in  arbitrary  order. 

a.  Any  process  of  type  llhl  can  be  regarded  as  a  repeated  choice  among  n  processes  oj 

tj'pe  10a. 

b.  Use  some  permutation  of  order  1,2,  .  .  .,  n  at  each  cycle. 
III7.     A^o  answer  theoretically  exact.     Nonstationary  process. 

15.  Use  infinitely  many  distinct  iteration  functions. 

a.  Use  polynomials  in  A  of  orders  1,  2,  3,  .    .    .    . 

1.  Generalized  Cheb\'shev  polynomials. 

2.  Other  pohTiomials. 

b.  Polynomials  in  other  matrices  connected  with  iterative  processes. 

c.  Monte  Carlo  method  (estimates  certain  population  means  related   to  A~\  b}-  sam- 

ples of  size  1,  2,  3,  .    .    .  )• 
1.     Based  on  Neumann  series  lOd. 
IV.     Miscellaneous  relevant  topics. 

16.  Analyses  of  errors. 

a.  Not  related  to  specific  method. 

1.  "Condition". 

2.  Stability  of  solution,  i.  e.,  variation  of  A~^  (or  A~^b),  as  A  varies  (or  A  and  b  vary), 

b.  Related  to  specific  method. 

1.  Truncation  errors. 

2.  Round-off  errors. 

A.  Worst  possible. 

B.  Probabilistic. 

S.     Control  of  blunders. 

17.  "Preconditioning"  (i.e.,  changuig  the  system  Ax=b). 

a.  By  enlarging  A. 

1 .  By  Gauss'  transformation. 

2.  By  homogenizing  the  system  (special  case  of  i). 

b.  By  premultiplication  of  A. 

1 .  By  A^  (normalization) . 

2.  By  some  polynomial  in  A. 

3.  By  som.e  approximation  to  A~^. 

c.  By  approximate  use  of  some  method  of  solution. 

1.  Orthogonalization. 

2.  Elimination. 


d.  By  taking  high  power  of  A  (after  17a;^). 

e.  By  successively  rotating  off -diagonal  elements  of  A  into  diagonal  (when  A  =  A'^). 

18.  Methods  for  punched-card  equipment. 

19.  Methods  for  automatic  digital  computing  machinery. 
19A.  Methods  for  analogue  computing  machinery. 

20.  Special  computing  tricks. 

a.  Cumulating  solution  as  ZAx^'^K 

1.     Using  few  significant  digits  at  once. 

b.  Getting  A~^  by  solving  n  systems  Ax^et  simultaneously  (operating  on  T). 

c.  Getting  A~^B  by  applying  to  B  the  row  operations  that  reduce  A  to  /. 

d.  If  PAn=I,  then  A-^^QP. 

e.  Splitting  real  parts  of  A  from  imaginary  parts  of  A. 

21.  jVIethods  for  accelerating  convergence,  applicable  to  many  problems  of  numerical  analysis. 

a.  Linear  processes. 

1.     Use  of  linear  combinations  of  sequence  of  a:'*\ 

b.  Nonlinear  processes. 

1.  Use  of  5^  process. 

2.  Use  of  generalized  b-  process. 

22.  Counts  of  operations,  time,  labor,  etc. 

23.  Methods  applicable  to  special  matrices. 

a.  Almost  triangular  matrices. 

b.  Matrices  of  large  order. 

V.     Contents  or  part  oj  contents  unknown. 

IV.  Cross  References  From  the  Outline  to  the  Bihliography 

In  this  section  the  subject  heading  (numbers  and  letters  like  llcl)  correspond  to  the  outline  in 
section  III.  Opposite  each  subject  heading  are  given  the  references  from  the  bibliography  (section  V) 
that  deal  with  the  subject.  There  is  an  inconsistency  in  the  presentation  about  which  the  user  should 
be  warned.  For  example,  KuNz  1947  treats  several  of  the  methods  2a,  2b,  2c,  .  .  .  ,  but  is  listed  only 
under  the  general  heading  2,  whereas  Bodewig  1947b  wdll  be  found  under  2a,  2b,  and  2c.  (The  un- 
availability of  Kuxz  1947  at  the  time  of  compiling  section  IV  caused  the  difference  in  treatment.) 

A  question  mark  after  an  entrj^  indicates  doubt  as  to  whether  the  title  deals  with  the  subject,  and 
does  not  query  the  date  as  such. 

0.  Bargmann-Montgomery-vom  Neumann  1946,  Bodkwig  1947b,  Engineering  Research  As- 

sociates 1950,  FoRSYTHE  1951,  Fox  1950a,  Frazer-Duncan-Coli.ar  1938,  Grossman  1950?, 
Hartree  1949b,  Householder  1919,  1950,  Kunz  1947,  Pipes  1948,  Salvadori  1948. 

a.  Geiringer  1949,  Hotelling  1943a,   1949,  Panov  1934,  Reich  1948,  ZuRMiiin.  1950. 

b.  DwYKR  1951,  Jensen  1944,  Kunz  1947,  Turing  1948,  Zurmuhl  1950. 

c.  Dunlap  1939,  DwYER  1941a,   1942,   1951,  Frame   1945,  Griffin   1931,   Higgins  1949,  Jennb 

1949,     Kantorovich-Krylov     1948,     International     Business     Machines     Corp.     1950, 
Southwell  1940,  1946,  Taussky  1951,  Tolley-Ezekiel  1927,  zur  Capellen  1948. 

la.  Adams  1915? 

1. 

a.  Bargmann-Montgomery-von   Neumann  1946,  Bodewig  1947a,  Cassina  1948,  Dickson  1922, 

Ingraham  1937,  Lkppert  1947,  Milnk  1949,  Ostkowski  1938,  Salvadori  1948,  Schmeidleh 
1949,  H.  ScHULz   1938. 

1.  AiTKEN  1937a,  Chi6  1853,  Collatz  l94Sa,  Dwyer  1951,  Goodwin  1950,  Jensen  1944,  Milne- 

Thomson   1941,  Pipes   1948,  Rice   1920,  Smith   1927,  Walthbr  1944,  Whittaker-Robinbon 
1924. 

b.  BissHOPP   1945,   Collar   1939,   Jkn.ne    1!»49,    Norli'nd    1!)40. 


2.  KuNz  1947. 

a.  AiTKEN  1932,  1935,  Andersen  1947,  Andree  1951,  Baetsle  1951,  Bargmann-Montgomery- 

voN  Neumann  1946,  Bauschinger  1901,  Bodewig  1947b,  Cassina  1948,  Cassinis  1946, 
CoLLATZ  1948a,  Couffignal  1944,  Deming  1928,  Dwyer  1941a,  1951,  Encke  1835,  For- 
SYTHE  1951,  Fox  1950a,  Fox-Huskey- Wilkinson  1948a,  Frazer-Duncan-Collar  1938, 
Froberg  1953,  Gauss  1826,  Harvard  Computation  Lab.  1946,  Herzberger  1949,  Herz- 
berger-Morris  1947,  Hoel  1941,  Householder  1949,  1950,  Jensen  1944,  1948,  Jurgens 
1886,  KoLMOGOROFF  1946?,  Margenau-Murphy  1943,  Mehmke  1930,  Milne  1949,  Mitchell 
1948,  Parkes  1950,  Polachek  1948,  Renner  1946,  Reynolds  1934,  Ricci  1949,  D.  F. 
Richardson  1946,  Salvadori  1948,  Scarborough  1950,  H.  Schulz  1938,  Seidel  1874, 
Spoerl  1943,  1944,  Verzuh  1949,  Walther  1944,  Whittaker-Robinsox  1924,  Willers 
1928,   1947,  WoRCH  1932,  Wren   1937,  Zurmuhl  1950. 

1.  Fox  1950a,  Goldstine-von  Neumann  1951,  Pipes  1948,  Turing  1948. 

A.  Black  1949,  Bruner  1947,  Doolittle  1878,  Dwyer  1941d,  1951,  Fox  1950a,  Kurtz  1936, 
Leavens  1947,  Nielsen-Goldstein  1947,  Satterthwaite  1944,  Tollay-Ezekiel  1927, 
Waugh-Dwyer  1945,  Rosser  1952. 

2.  Dodgson  1866,  Dwyer  1951,  Waugh-Dwyer  1945,  Rosser  1952. 

b.  AiTKEN  1932,  Bodewig  1947b,  Boschan  1946,  Cochran  1938?,  Collatz  1948a,  Dwyer  1951, 

Frazer  1947,  Householder  1949,  1950,  Jossa  1938?,  1940?,  Morris  1946,  1947,  Neville 
1948,  Turing  1948. 

1.  Fox  1950a,  Fox-Huskey- Wilkinson  1948a,  Turing  1948. 

c.  Andersen   1947,   Baetsl6   1951,   Banachiewicz   1951,  Bodewig   1947b,   1950,   Cassina   1948, 

Cassinis  1946,  Crout  1941,  Dwyer  1951,  Forsythe  1951,  Fox  1950a,  1950b,  Fox-Huskey- 
Wilkinson  1948a,  Herzberger  1949,  Householder  1949,  1950,  Istituto  per  le  Applica- 
ziONi  DEL  Calcolo  #4  (date  unknown),  Jensen  1944,  Milne  1949,  Picone  (date  unknown), 
Ricci  1949,  Turing  1948,  Waugh-Dwyer  1945,  ZuRMtJuL  1949,  1950. 

1.  AspLUND    1945,    Banachiewicz    1938a?,    1938b,    Benoit    1924,   D.  B.  Duncan-Kenney    1946, 

Dwyer  1942,  1944,  1945,  Istituto  per  le  Applicazioni  del  Calcolo  #3  (date  unknown), 
Jensen  1944,  Kunz  1947,  Laderman  1948,  Rubin  1926. 

d.  Dwyer    1951,    Fox    19o0a,    Fox-Huskey- Wilkinson    1948a,    Frazer-Duncan-Collar    1938, 

Herzberger  1949,  Herzberger-Morris  1947,  Waugh-Dwyer  1945,  Zurmuhl  1950. 

e.  Andree  1951,  Baetsle  1951,  Dwyer  1951,  Forsythe  1951,  Fox  1950a,  Hoel  1941,  Verzuh 

1949. 

f.  Backman  1946,  Banachiewicz  1937b,  1937c,   1942,   1951,  Kamela  1943. 
3. 

a.  Basile  1949,  Kunz  1947? 

1.  Bodewig  1947b,  Fox-Huskey- Wilkinson  1948a,  Frazer-Duncan-Collar  1938,  Jxjrgens  1888. 

2.  Clasen  1888,  Dreyer  1943,  Forsythe  1951,  Fox  1950a,  Householder  1949,  Jensen  1944, 

Jordan  1920,  Petrie  1953,  Samuelson  1950,  Turing  1948,  Volta  1950,  Whittaker-Robin- 
SON  1924,  WoRCH  1932. 

S.  Forsythe  1951. 

b.  Bargmann-Montgomery-von  Neumann    1946,    Cassinis   1946,    Forsythe    1951,   Fox   1950a, 

Householder  1950,  Kunz  1947?,  Picone  (date  unknown),  Ricci  1949,  Roma  1946,  1947, 
E.  Schmidt  1908,  Shreider  1951,  Turing  1948,  Unger  1951. 


\\a. 


up. 


IIIc 


4.  Banachiewicz  1937a,  1937b,  1937c?,  Bargmann-Montgomery-von  Neumann  1946,  Bodewig 

1947a,  1947b,  Boltz  1923,  Collatz  1948a,  W.  J.  Duncan  1944,  Forsytiie  1951,  Fox  19o0a, 
Frazer-Duncan-Collar  1938,  (Gram  1883),  Guttman  1946,  Hotellixg  1943a,  1943b,  1949, 
Ingraham  1937,  Jossa  1938,  1940,  Jensen  1944,  Kruger  1905,  Kunz  1947?,  Morris  1946, 
1947,  OsTRowsKi  1938,  Pipes  1941,  Saibel  1944,  Schur  1917,  Turetsky  1951,  Waugh  1945, 
Woodbury  1950. 

a.  (Many  of  references  under  4.) 

b.  Neville  1948. 

5.  Bodewig  1947b,  Forsythe  1951,  Polachek  1948,  Runge-Konig  1924,  Salvadori  1948,  Thomson 

1878,  Turing  1948,  Zurmuhl  1944,  1950. 


a.  Dwyer  1951. 

/.  HOTELLING  1943a. 

b. 

1.  Bingham  1941,  Dwyer  1951,  Frame  1949,  Pipes  1948. 

S.  Forsythe  1951,  Freeman  1943,  Milne  1951. 

c.  Freeman  1943,  R.  J.  Schmidt  1941. 

d.  Hestenes  1951,  Lanczos  1951,  Stiefel  1951,  1952,  Rosser  1953. 

a.  Lewy  (unpublished),  Robinson  1951. 
b. 

1.  Dantzig  1951. 

2.  Sherman  1951,  Sherman-Morrison  1949. 


9. 

0.  Hestenes  (unpublished). 


10.  Argelander  1844,  Bauschinger  1901,  Cherepkov  1930,  Grossman  1948,  Kantorovich  1939, 

Kunz  1947,  Waddell  1910,  Zvlev  1939. 

a.  Aitkkn  1950,  Cesaui  1937a,  1937b,  Collatz  19.50b,  Fuazer-Duncan-Collar  1938,  Forsythe 

1951,  Geikingkk  1949,  Hotkllinc  1943,  Householder  1949,  Milne  1951,  Morris  1935?, 
Pipes  1948,  Reich  1948,  Wittmeyer  1936b. 

b.  An^r  1920,  Black  (date  unknown).  Brand  1935?,  Cassinis  1946,  Chen  1944?,  Collatz  1942, 

1948a,  1949,  1950a,  1950b,  Cro.ss  1932,  Forsythe  1951,  Fox  1950a,  Frazer-Duncan-Collar 
1938,  Geiringer  1949,  Householder  1949,  19.50,  Ivanov  1939,  Jacobi  1845,  Kormes  1943?, 
1947?,  Morris  1947,  Ri;N(;e-Komg  1924.  Schott  1855,  Stein-Rosenberg  1948,  von  Mises- 
Pollaczek-Geirin(;er  1929,  YouN(i  19.50. 


c.  AiTKEN  1950,  Bowie  1950,  Cassinis  1946,  Collatz  1942,  1948a,  1949,  1950a,  1950b,  Forsythe 

1951,  Fox  1950a,  Frankel  1950,  Frazer-Duncan-Collar  1938,  Gatto  1949,  Geiringer 
1949,  HoTELLiNG  1933,  1943a,  Householder  1949,  1950,  Ivanov  1939,  Kantorovich-Krylov 
1948,  LiEBMANN  1918,  Mehmke  1892,  Mehmke-Nekrassof  1892?,  Miller  1947,  Morris 
1935,  1947,  Oldenburger  1940b,  Pipes  1948,  Pollaczek-Geiringer  1928,  Reich  1948,  1949, 
Runge  1899,  Salvadori  1948,  Sassbnfeld  1951,  Schmeidler  1949,  Seidel  1862,  1874,* 
Shortley-Weller  1938,  Shortley-Weller-Fried  1940,  Snyder-Livingston  1949,  P.  Stein 
1951,  Stein-Rosenberg  1948,  von  Mises-Pollaczek-Geiringer  1929,  Whittaker-Robin- 
soN  1924,  WiLLERS  1928,  1947,  Young  1950,  Zurmuhl  1950. 

d.  Biezeno  1924,  Bodewig  1947b,  Forsythe  1951,  Frankel  1950,  Geiringer  1949,  Hellinger- 

ToEPUTZ  1924,  HoLLEY  1951,  Hotelling  1943a,  Householder  1949,  Milne  1951,  Newing 
1941,  Pipes  1948,  Plunkett  1950,  Quade  1947,  Richardson  1910,  Schmeidler  1949,  von 
Mises-Pollaczek-Geiringer  1929,  Walsh  1920,  Waugh  1950. 


1.  Gauss  1826,  Geiringer  1942,  1949,  Hertwig  1912,  Mehmke  1892. 

f.  Bodewig  1947b,  Cimmino  1938,  Forsythe  1951,  Householder  1949. 

g.  Frankel  1950. 
11. 

a  (comprehensive). 

Forsythe  1951,  Householder  1949,  Rosser  1949,  1950,  M.  Stein  1952. 

{1  &  6).     Birman  1950?,  Householder  1950,  Kantorovich  1945,  1947,  Mysovskikh  1950?,  Pipes  1948, 
Temple  1939. 

(2  &  5).     Booth  1949,  Cauchy  1847,  Curry  1944,  Forsythe-Motzkin  1950,  1951a,  1951b,  Hartree  1948, 
1949b,  Kantorovich  1948,  Kantorovich-Krylov  1948,  Reich  1948. 

(5  &  5).  Forsythe-Motzkin  1950,  1951b. 

{3  &  6).  Kantorovich  1947. 

{2  &  4&5).  Cauchy  1847,  M.  Stein  1952. 

4.  Hartree  1948?,  M.  Stein  1952. 

b. 

1.  Black-Southwell  1938,  Bowie  1947?,  Fox  1947,  Hendrikz  (date  unknown)?,  Menzies  (date 

unknown),  Nikolaeva  1949,  L.  Wright  1943. 

{A  &  F  &  I).  Black  1938,  Black  (date  unknown),  Forsythe  1951,  Fox  1948,  1950b,  Gaskell  1943,  House- 
holder 1950,  Milne  1951,  Reich  1948,  Southwell  1940,  1946. 

{D  &  G  &  J).     Householder  1949. 

(B&  F&  /).  Dedekind  1901,  Fox  1948,  Gauss  1823,  Gerling  1843,  JtJROENS  1886,  Rainsford  (date  un- 
known), ScHAEFER  1927,  ScHOTT  1855,  Southwell  1940,  1946,  Temple  1939,  Thompson  (date 
unknown),  Whittaker-Robinson  1924,  Zurmuhl  1950. 


L.     Fox  1948,  Gerling  1843,  Southwell  1940,  1946. 
(C  &  F  &  J>.     Fox  1950a,  Householder  1949,  Seidel  1874,  Synge  1944. 

2.  Black  (date  unknown).  Fox  1948,  1950b,  Southwell  1940,  1946. 


•See  footnote,  p.  5. 
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1.  Bargmann-Montgomery-von  Neumann  1946,  Bodewig  1947b,  Fox  1950a,    Frazer-Ditncan- 

CoLLAR   1938,   Hotelling   1943a,    1949,   Householder   1949,   Kunz    1947?,    Lonseth    1949, 
Kantorovich  1949,  Neville  1948,  Pipes  1948,  G.  Schulz  1933,  Turing  1948,  Ullman  1944. 

2.  Bargmann-Montgomert-von  Neumann  1946. 

3.  Bodewig  1947b,  Schroder  1870,  Ullman  1944. 

12.  LfDsTERNiK  1947,  Shortley-Weller  1938,  Shortley-Weller-Fried  1940. 

a.  FORSYTHE  1951. 

b.  Hestenes  (unpublished). 

C.  FORSYTHE-MOTZKIN   1951b. 

d.  (References  under  llb^.) 

e.  FoRSYTHE  1951,  Richardson  1910. 

f.  Fox  1950b,  Milne  1951. 


13. 


14. 


a  (comprehensive).      Householder  1949. 

{1  &  4)-  DE  LA  Garza  1951,  Householder  1949,  Rosser  1949. 

(2  &  6).  Bodewig  1947b,  Bottema  1950,  Kaczmarz  1937,  Rosser  1949,  Tompkins  1949. 

(i  &  5).  Frankel  1950,  Householder  1950. 

(/  &  3  &  5).  Frankel  1950. 

(B  &  4).  Householder  1949. 
b. 

1.  AlTKEN  1950. 

c.  BtJCKNER  1950,  Geiringer  1949,  Richardson  1910. 
/.  Aitken  1950,  Rosser  1949,  1950. 

d.  Kelley-Salisbi[ry  1920. 

a.  Hestenes  (unpublLshcd). 

h.  Geiringer  1949,  Reich  1948. 


III7. 

15. 


a.  Flanders-Shortlky  1950. 

b. 

c. 
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1.  Forsythe-Leibler  1950,  Opler  1951,  Swift-Tikson  1951,  Todd  1951a,  1951b,  Wasow  1952. 

IV. 

16.  Dwyer  1953,  Opitz-\Yillers  (date  unknown),  Pirlet  1909,  H.  Schulz  1938. 

a.  Hertwig  1905?,  Price  1951,  Sherman-Morrison*  1950. 

1.  Hartree  1948,  JtJRGExs  1888,  Taussky  1949,  1950,  Todd  1949a,  1949b,  Turing  1948. 

2.  Barg.mann-]Montgomery-von  Neumann  1946,   Berkson  1936,  Bartlett  1951,   Blumenthal 

1914,  CoLLATZ  1949,  W.  E.  Deming  1937,  Dwyer  1951,  Etherington  1932,  Forsythe  1951, 
HoTELLixG  1943a,  Janet  1920a,  1920b,  Lonseth  1942,  1944,  1947,  Milne  1949,  Morgen- 
STERN-WooDBURY  1950,  MouLTON  1913,  1936,  OsTROwsKi  1937a,  1937b,  1950,  Redheffer 
1948,  RoEssLER  1936,  Scarborough  1950,  Tuckerman  1941,  Turing  1948,  Walsh  1920, 
WiLLERS  1928,  1947,  Wittmeyer  1934,  1936a,  Woodbury  1949,  1950,  Zurmuhl  1950. 

b.  HoTELLiNG  1949,  Householder  1949,  Leavens  1947. 

1.  CoLLATZ  1950b,  Wittmeyer  1936b. 

2.  Bargmann-Montgomery-von  Neumann  1946,  Dwyer  1951,  Goldstine-von  Neumann   1951, 

Neville  1948,  Parkes  1950,  Polachek  1948,  Reich  (date  unknown),  Reich  1948,  Salva- 
DORi  1948,  Satterthwaite  1944,  Scarborough  1950,  Tuckerman  1941,  Turing  1948, 
Waugh  1950. 

A.  HoEL  1940,  Hotelling  1943a,  1943b. 

B.  HoEL  1940,  Hotelling  1943a,  Ullman  1944. 
S.              Dwyer  1951,  Gauss  1823. 

17. 
a. 

1.  Forsythe-Motzkin  1952,  Gauss  1823,  Gerling  1843,  ZuRMtJHL  1950. 

b. 

1.  Bargmann-Montgomery-von  Neumann     1946,     Black-Southwell     1938,     Cassinis     1946, 

Collatz  1942,  1948a,  Hotelling  1943a,  Householder  1950,  Kantorovich  1948,  Milne 
1951,  Seidel  1874,  Taussky  1949,  1950,  Temple  1939,  von  Mises-Pollaczek-Geiringer 
1929,  Zurmuhl  1950. 

2.  Bodewig  1947b,  Cesari  1937b. 
S.              Satterthwaite  1944. 

c. 

1. 

2.  Bodewig  1947b,  Bowie  1950,  Jtjrgens  1886. 

^-  Geiringer  1949,  Hotelling  1936a,  1943a. 

e-  Jacobi  1845,  Seidel  1874. 

Akushsky  1946a,  1946b,  1946c,  Alt  1946,  Basile  1949,  Eckert  1940,  Engineering  Research 
Associates  1950,  Flanagan  1940,  Forsythe  1951,  Fox-Huskey-Wilkinson  1948b,  Gross- 
man 194S,  Hartley  1946,  IBM  1950  (complete),  (IBM  educational  forums),  Kormes  1943, 
Leppert  1947,  Opler  1951,  Petrie  1953,  Renner  1946,  Sherman  1951,  Snedecor  1928, 
Tucker  1940,  Verzuh  1949. 
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18. 


19.  Engineering  Research  Associates  1950,   Froberg   1953,   Goldstine-von  Neumann    1951, 

Hartree  1949b,  Harvard  Computation  Lab.  1946,  McPherson  1948,  Mitchell  1948. 
RossER  1950,  Snyder-Livingston  1949,  Stiefel  1951,  1952. 

19A.  Adcock    1948,    Anonymous    1934,    Berry- Wilcox-Rock- Washburn    1946,    Bowie    (date   un- 

known), Brown  1947,  Dennis  1948,  Engineering  Research  Associ.\tes  1950,  Fairthorne 
1944,  Frame  1945,  Fuchs  1914,  Gorushkin  1948,  Gradstein  1947,  Hardy-Dench  1948, 
Haupt  1950,  Hazen-Schurig-Gardner  1930,  Korn  1949,  Kron  1945,  Mallock  1933,  jMany 
1950,  McCann  1949,  Meerovich  1947,  Muirhe.\d  1912,  Murray  1947,  1949,  Nabauer  1910, 
Parker  1941,  1945,  Picchi  1948,  Pipes  1948,  Proshko  1947,  Samssonow  1943,  Schumann 
1940,  Spangenberg-Walters-Schott  1949,  Wilbur  1936,  zur  Capellen  1948. 


20. 


a.  Black  (date  unknown),  Dedekind  1901,  Cross  1932.  Gauss  1823,  Jacobi  1845,  Salvadori  1948. 
1.  Forsythe  1951,  Fox  1948,  1950a?,  1950b,  Southwell  1940,  1946,  Zurml'hl  1950. 

b.  Aitken   1932,   Albert   1941,   Dwyer   1951,   Fisher   1938,   Forsythe   1951,   Frazer-Duncan- 

CoLLAR  1938,  HoEL  1941,  HoTELLiNG  1943a,  Householder  1950,  Milne  1949,  G.  Schulz  1933. 

c.  Burgess  1916,  Dwyer  1941c,  1951,  Forsythe  1951. 

d.  Forsythe  1951. 

e.  Fox  1950a,  Goodwin  1950,  Pipes  1941,  Zurmxjhl  1950. 


21. 


1.  Forsythe  1951,  Fox  1948,  Southwell  1940?,  1946? 
b. 

1.  Aitken  1926,  1937b,  1950,  Forsythe  1951,  Fox  1950a,  Samuelson  1945,  Shanks  1949. 

2.  Forsythe  1951,  Samuelson  1945,  Shanks  1949. 

22.  Bargmann-Montgomery-von  Neumann  1946,  Bodewig  1947b,  Cassina  1948,  Collatz  1948a, 

Frankel  1950,  Heck  1946,  Kamela  1943,  Parkes  1950,  Ricci  1949. 

23. 

a.  Bisshopp  1945,  Good  1950. 

b.  Goldstine-von  Neumann  1951,  Jenne  1949,  Kerkhofs  1947. 

V.  Ballantine  1931,  W.  E.  Deming  1935,  D.  B.  Duncan-Kenney  1946,  Dunlap  1939,  Dwyer 

1941b,  1946,  Encke  1835,  Flanagan  1940,  Fox  1947,  1950a,  Fox-Huskey- Wilkinson  1948b, 
Griffin  1931,  Grossman  1948,  1950,  Halperstadt  1914,  Hartley  1946,  Heck  1946,  Hen- 
DRiKZ  (date  unknown),  Horst  1941a,  Hrtska  1943,  Jenne  1949,  Jordan  1920,  Jossa  1938, 
1940,  Kantorovich-Krylov  1941,  Kerkhofs  1947,  Kolmogoroff  1946,  Kormes  1943, 
1945,  LiusTERNiK   1947,  Magnier  1948,  Oppokov  1939,  Panov  1938,  Picone  1940,  Reich 

(date  unknown),   Schaefer   1927,   H.   Schllz    1938,   Shaw    1946,   Turton   1945,    Wegner 

(date  unknown). 

V.  Bibliography,  With  Cross-Refcrenees  to  the  Outline 

In  this  section  arc  given  approximately  450  titles  on  the  solution  of  Imear  equations,  taken  from  the 
compiler's  card  file  on  numerical  matrix  methods.  The  card  file  has  been  collected  during  the  past  two 
years  as  part  of  an  investigation  at  the  Xational  Bureau  of  Standards,  Los  Angeles,  of  old  and  new 
matrix  methods  suited  to  automatic  digital  computers. 
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The  bibliography  is  most  nearly  complete  in  recent  titles  dealing  with  mathematical  methods  for 
obtaining  A''  or  A'^b,  where  ^  is  a  nonsingular  square  matrix.  The  bibliographies  of  Bodeavig  1947b, 
DwYER  1951,  Harvard  Computation  Laboratory  1946,  Higgins  1949,  Hotelling  1943,  House- 
holder 1949,  Kantorovich  1948,  Kantorovich-Krylov  1948,  and  Ostrowski-Todd-Todd  1949  proved 
invaluable  in  starting  and  augmenting  the  list.  Original  papers  were  consulted  when  reasonably  obtain- 
able, and  their  references  were  added  to  the  card  file.  Certain  papers  which  appear  to  be  covered  in 
subsequent  books  have  been  omitted;  some  titles  containing  lists  of  such  papers  are  given  under  heading 
Oc  of  section  IV. 

Because  much  of  the  research  on  solving  linear  equations  has  been  ancillary  to  the  solutioa  of 
numerical  problems  in  pure  mathematics,  statistics,  physics,  astronomy,  geodesy,  psychology,  economics, 
engineering,  etc.,  many  good  references  are  from  these  diverse  fields.  The  present  bibliography  is  less 
complete  in  these  related  fields,  some  of  which  have  their  own  bibliographies  (e.  g.,  Higgins  1949  on 
numerical  solution  of  partial  difi'erential  equations) . 

There  are  several  mathematical  fields  which  are  related  to  our  subject,  but  which  have  not  been 
included  here.  Three  of  these  are  Imown  to  be  subjects  for  bibliographies  now  in  preparation: 
Motzkin  1951  (on  linear  mequalities  and  applications);  Taussky  1951  (on  bounds  for  eigenvalues); 
Schwerdtfeger  1951  (on  iteration  in  general;  listed  at  end  of  bibliography). 

At  the  last  minute  an  effort  was  made  to  include  some  representative  titles  on  analogue  machinery 
for  solving  linear  equations,  starting  with  references  m  Frame  1945  and  Tryon  1951.  However,  in 
regard  to  analogue  machinery  the  bibliographies  of  Engineering  Research  Associates  1950  and  zur 
Capellen  1948  are  far  more  extensive  than  the  few  titles  included  below.  Also,  the  bibliography  of 
International  Business  Machines  1950  (mostly  reproduced  in  Engineering  Research  Associates 
1950)  is  more  complete  on  uses  of  IBM  equipment  for  matrix  problems  than  the  present  bibliography. 

The  numbers  in  brackets  after  each  title  in  the  bibliography  refer  to  the  outline  in  section  III,  and 
summarize  the  contents  of  the  reference.  The  notation  llhl{Bd;F<&I)  means  that  the  paper  treats  the 
subcase  of  subject  llhl,  in  which  B,  F,  and  /  are  simultaneously  fulfilled  (logical  product). 

A  circle  (°)  is  placed  before  each  title  which  the  compiler  has  not  examined  personally,  insofar  as 
he  can  remember. 

The  abbreviations  MR,  Fs,  Zb,  SA  after  titles  refer  to  abstract  journals: 

M  R = Mathematical  Reviews ; 

Fs    =Jahrbuch  iiher  die  Fortschritte  der  Mathematik ; 
Zb    —  Zentralblatt  der  Mathematik  und  ihre  Grenzgebiete; 
SA  =  Science  Abstracts,  Section  A. 

References  to  abstract  journals  are  listed  when  they  happen  to  be  known;  there  is  no  miiform  policy. 

The  following  journal  abbreviation  has  been  used  in  the  references:  MTAC,  Mathematical  Tables 
and  Other  Aids  to  Computation. 

Attention  is  invited  to  the  supplementary  bibliographies  at  the  end,  containing  papers  noted  too 
late  for  inclusion  in  the  main  list. 

Adams,  Oscar  S.      1915:     Application  of  the  Theory  of  Least  Squares  to  the  Adjustment  of  Triangulation,  USCGS  Special 

Publ.  No.  28,  U.  S.  Govt.  Printing  Office.     [la?]. 
°Adcock,  W.  a.     1948:     "An  automatic  simultaneous  equation  computer  and  its  use  in  solving  linear  equations,"  Rev. 

Sci.  Inst.  19,  181-187.     [19A]. 
"AiTKEN,  A.  C.     1926:     "On  Bernoulli's  numerical  solution  of  algebraic  equations,"  Proc.  Roy.  Soc.  Edinburgh  46,289- 

305.     [21bi]. 
AiTKEN,  A.  C.     1932:     "On  the  evaluation  of  determinants,  the  formation  of  their  adjugates,  and  the  practical  solution 

of  simultaneous  linear  equations,"  Proc.  Edinburgh  Math.  Soc.  (2)  3,  207-219.     [2a,  2b,  20b]. 
AiTKEN,  A.  C.     1937a:     "Studies  in  practical  mathematics.     I.  The  evaluation,  with  applications,  of  a  certain  triple 

product  matrix,"  Proc.  Roy.  Soc.  Edinburgh  57,  172-181.      [la,  2e]. 
AiTKEN,  A.  C.     1937b:     "Studies  in  practical  mathematics.     II.  The  evaluation  of  the  latent  roots  and  latent  vectors 

of  a  matrix,"  Proc.  Roy.  Soc.  Edinburgh  57,  269-304.      [21bl]. 
AiTKEN,  A.  C.     1945:     "Studies  in  practical  mathematics.     IV.  On  linear  approximation  by  least  squares,"  Proc.  Roy. 

Soc.  Edinburgh  A62,  138-146.     [2a]. 
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AiTKEN,  A.  C.      1950:     "Studies  in  practical  mathematics.     V.  On  the  iterative  sokition  of  a  system  of  linear  equations," 
Proc.  Rotj.  Soc.  Edinburgh  A63,  52-60.     [10a,  10c,  13c/,  21b/]. 
°Akushsky,  I.  J.      1946a:     "Numerical  solution  of  the  Dirichlet  equation  with  the  aid  of  perforated  card  machines," 

C.  R.  (Doklady)  Acad.  Sci.  URSS  52,  375-378.     [18]. 
"Aktjshsky,   I.  J.      1946b:     "The  four-counter  scheme  of  solution  of  Dirichlet's  problem  by  means  of  punched-card 

machines,"  C.  R.  {Doklady)  Acad.  Sci.  URSS  54,  659-662.     [18]. 
°Akushsky,  I.  J.     1946c:     "On  numerical  solution  of  Dirichlet  problem  on  punched-card  machines,"  C.  R.  (Doklady) 

Acad.  Sci.  URSS  54,  755-758.     [18]. 
Albert,  A.  A.     1941:     "A  rule  for  computing  the  inverse  of  a  matrix,"  Am.  Math.  Monthly  48,  198-199  (MR  2,  243). 

[20b]. 
Alt,  Franz  L.      1946:     "Multiplication  of  matrices,"  MTAC2,  12-13.     [18]. 
"Andersen,  Einar     1947:     "Solution  of  great  systems  of  normal  equations  together  with  an  investigation  of  Andrae'a 
dot-figure.     An  arithmetical-technical  investigation,"    Mem.  Inst.  G^odisique  Danemark  (Geodaetisk  Inst.   Skr.)    (3), 
11,  65  pp.  (MR  9,  622).     [2a,  2c]. 
Andree,  R.  V.      1951:     "Computation  of  the  inverse  of  a  matrix,"  Am.  Math.  Monthly  58,  87-92.     [2a,  2e,  20c]. 
°Aner,  H.     1926:     "Ausgleichung  durch  Anwendung  des  arithmetischen  Mittels,"  Zeitschrift  fur  Vermessungswesen  55, 

65-77.     [10b]. 
"Anonymous?     1934:     "The   Mallock  electrical  calculating  machine,"   Reprint  from  Engineering   (London),  June  22, 
1934,  8  pp.     [19A]. 
Argelander,  F.  W.  a.     1844:     "Ueber  die  Anwendung  der  ISIethode  der  kleinsten  Quadrate  auf  einen  besondern  Fall," 
Astr.  Nachr.  21,  No.  491,  163-168.     [10.    Method  unclear.] 
°Asplund,  Lars     1945:     "tJber  einige  Methoden  fiir  die  Ausgleichung  geodatischer  Netze,"  Rikets  Allmdna  Kartverk, 

Meddelande  No.  5,  Stockholm.     [2c/]. 
Backman,    Gaston     1946:     "Rekursionsformeln   zur   Losung   der    Normalgleichungen   auf   Grand   der    Krakovianen- 

methodik,"  Arkiv  for  Matematik,  Astronomi  och  Fysik  33A,  1-14.     [2f]. 
Baetsl6,  p.  L.      1951:     "Systemisation  des  calculs  numeriques  de  matrices,"  Bulletin  Giodisique,  22-41.     [2a,  2c,  2e]. 
"Ballantine,   J.    P.     1931:     "Numerical  solutions   of  linear  equations  by   vectors,"  Amer.   Math.    Monthly  38,   275- 

277.     [V], 
°Banachiewicz,   T.      1937a:     "Zur  Berechnung  der  Determinanten,  wie  audi  der  Inversen,  imd  zur  darauf  basierten 

Auflosung  der  Systeme  linearer  Gleichungen,"  Acta  Astronomica,  3,  41-67.     [4]. 
"Banachiewicz,  T.     1937b:     "Calcul  des  determinants  par  la  methode  des  cracoviens,"  Bull.  Intern,  de  V Acad.  Polonaise, 

Sirie  A,  Sci.  Math.,  109-120.     [2f,  4]. 
°Banachiewicz,  T.     1937c:     "Sur  la  resolution  num^rique  d'un  systeme  d'equations  lin^aires,"  Bull.  Intern,  de  I'Acad. 

Polonaise,  S6rie  A,  Sci.  Math.,  350-354.     [2f,  4?]. 
°Banachiewicz,  T.     1938a:     "Principes  d'une  nouvelle  technique  de  la  methode  des  moindres  carr6s,"  Bull.  Intern,  de 

I'Acad.  Polonaise,  S6rie  A,  Sci.  Math.,  134-135.      [2c/?]. 
°Banachiewicz,  T.      1938b.     "M6thode  de  resolution  numerique  des  equations  lin^aires,  du  calcul  des  determinants  et 
des  inverses,  et  de  reduction  des  formes  quadratiques,"  Bull.  Intern,  de  I'Acad.  Polonaise,  Serie  A,  Sci.  Math.,  393-404. 
Reprinted  in  Cracow  Observatory  Reprint  22.     [2c/]. 
Banachiewicz,  T.      1942:     "An  outline  of  the  Cracovian  algorithm  of  the  method  of  least  squares,"  Astro7i.  Journal, 

38-41.     [2f]. 
Banachiewicz,   T.      1951:     "Resolution  d'un  systeme  d'equations  lineaires  algebriques  par  division,"   Enseignement 

Math.  39,  (1942-1950),  34-45.     [2c,  2f]. 
Bargmann,  v.,  Montgomery,  D.,  and  von  Neumann,  J.     1946:     "Solution  of  linear  systems  of  high  order,"  Report 
prepared  for  the  Bureau  of  Ordnance  (Contract  NORD-9596)  (25  Oct.  1946),  86  pp.     [0,  1,  2a,  3b,  4,  lie/,  Uc^,  16a^, 
16b^,  17b/,  22]. 
*°Bartlett,  M.  S.      1951:     "An  inverse  matrix  adjustment  arising  in  discriminant  analysis,"  Ann.  Math.  Stat.  22,  107-111. 

[16a2]. 
°Basile,  R.      1949:     "Resolution  de  systemes  d'6quations  lineaires  algebriques  et  inversions  de  matrices  au  moyen  des 
machines  de  mecanographie  comptable.     Complement  pratique  par  R.  Janin,"  Office  National  d' Etudes  et  de  Re- 
cherches  Aironautiques,  Paris,  publ.  no.  28,  v  +  21  pp.  (MR  11,  692).     [3a?,  18]. 
Bauschinger,  Julius     1901:     "Au.sgleichungsrcchnung,"  Enc.  d.  Math.  Wiss.  ID  2,  786-798.     [2a,  10]. 
Benoit,  Co.mmandant     1924:     "Note  sur  une  m6thode  de  resolution  des  equations  normales  etc.,"  International  Geo- 
detic and  Geophysical  Union,  Association  of  Geodesy,  Bulletin  Giodisique  (Toulouse),  no.  2,  67-77.      (Translation  by 
Rainsford  available  in  UMT  file  of  MTAC.     See  MTAC,  April  1951.)     [2c/]. 
Berkson,  Joseph     1936:     "Significant  figures  in  statistical  constants,"  Science  84,  437.     [16a^]. 
Berry,  Clifford  E.     1945:     "A  criterion  of  convergence  for  the  classical  iterative  method  of  solving  linear  simultaneous 

equations,"  Ann.  Math.  Stat.  16,  398-400.     [10c]. 
"Berry,  C.  E.,  Wilcox,  D.  F].,  Rock,  S.  M.,  and  Washburn,  H.  W.     1946:     "A  computer  for  solving  linear  simultaneous 

equations,"    ./.    Appl.    Phys.    17,    262-272.     [19A]. 
°Bikzeno,  C.  B.     1924:     "Zeichnerische  Ermittlung  der  elastischen  Linie  eines  federnd  gesttitzten,  statisch  unbestimmten 

Balkens,"   Zeitschr.  f.   Angew.   Math.    Mech.   4,  93-102.     [lOd]. 
"Bingham,  M.  D.     1941:     "A  new  method  for  obtaining  the  inverse  matrix,"  J.  Amer.  Stat.  Assoc.  36,  530-534.     [6b/]. 
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°BiRMAN,   M.  SH.   1950:     "Eiiiige  Abschatzungen  ftir  die  Methode  des  schnellsten  Abstiegs"  (Russian),  TJspekhi  Maiem. 
Nauk  5,  no.  3   (37),  152-155.     (Zb  38,  80).     [11a  (1   &  6)]. 
BissHOPP,  K.  E.     1945:     "The  inverse  of  a  stiffness  matrix,"  Quart.  Appl.  Math.  3,  82-84.     [lb,  23a]. 
Black,  A.  N.     1938:     "The  method  of  relaxation  applied  to  survey  problems,"  Empire  Survey  Review  4  (no.  29),  406- 

413.'   [nh\{A  &  F  &  /)?]. 
Black,  A.  X.     (date  unknown):     "Approximate  methods  of  solving  normal  equations,"  Empire  Survey  Review  7  (no. 

52),  242-245.     (Probably  c.  1942.)      [10b,  llbi(^  &  F  &  I),  llb^,  20a]. 
Black    A.  N.     1949:     "Further  notes  on  the  solution  of  algebraic  linear  simultaneous  equations,"  Quart.  J.  Mech. 

Appl.  Math.   (Oxford)   2,  321-324   (MR  11,  743).     [2a/^]. 
Black    A.  N.,  and  Southwell,   R.   V.     1938:     "Relaxation  methods  applied  to  engineering  problems.     II.     Basic 
theory,  with  application  to  surveying  and   to  electrical  networks,    and  an  extension  to  gyrostatic  systems,"  Proc. 
Roy.  '^Soc.  A  164,  447-467.     [llbi,  17b/]. 
"Blumenthal,  O.     1914:     "Uber  die  Genauigkeit  der  Wurzeln  linearer  Gleichungen,"  Zeit.  Math.  u.  Physik  62,  359-362 

(Fs  45,   174).     [16ag]. 
°BoDEWiG,    E.     1947a:     "Comparison  of  some  direct   methods   for   computing  determinants   and   inverse   matrices," 
NederL  Akad.   Wetensch.,  Proc.  50,  49-57  (MR  8,  407).     [la,  4]. 
BoDEw^iG,   E.     1947b:     "Bericht  tiber  die  verschiedenen   Methoden  zur  Losung  eines  Systems  linearer  Gleichungen 
mit  reellen  Koeflfizienten.     I,  II,  III,  IV,  V,"  Nederl.  Akad.  ire^ensc/i.,  Proc.  50,  930-941,  1104-1116,  1285-1295  and 
51,  53-64,  211-219.     Same  articles  in  Indagationes  Math.  9,  441-452,  518-530,  611-621   (1947),  and  10,  24-35,  82-90. 
[0,  2a,  2b,  2c,  3a/,  3b,  4,  5,  10a,  10b,  10c,  lOd,  lOf,  lie/,  llc5,  13a  {2  &  6),  17b^,  \'Ic2,  22]. 
BoDEwiG,  E.,  and  ZuRMtJHL,  R.     1950:     "Zu  R.  Zurmlihl:     Zur  numerischen  Auflosung  linearer  Gleichungssysteme 
nach  dem  Matrizenverfahren  von  Banachiewicz.  Z.  angew.   Math.    Mech.  29    (1949)    76-84,"    Zeitschr.   f.    Angew. 
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2.  Siimiltaneoiis  Systems  of  Equations 

A.  INI.  Ostrowski  ^ 

2.1 .  In  the  theory  of  systems  of  n  linear  equations  with  n  unknowns,  the  problem  of  existence  of 
)lutions  does  not  present  any  difficulties  whatever,  as  soon  as  the  determinant  can  be  computed  and  is 
ot  zero.  On  the  other  hand,  the  interdependence  between  the  solutions  and  the  coefficients  can  be 
reated  in  introducing  different  expressions  corresponding  to  different  assumptions  about  the  norm  in  the 
ector  space  in  question  [1,  2,  3).  The  theory  of  a  system  of  n  nonlinear  equations  with  n  unknowns 
ppears  from  the  beginning  as  essentially  more  difficult,  because  not  even  the  existence  of  solutions  can 
e  treated  m  a  corresponding  way.  If  the  Jacobian,  the  analog  of  the  determinant,  is  not  zero,  we  can 
nly  say  something  about  the  continuation  of  the  originally  given  solution  in  a  sufficiently  small  neigh- 
ourhood. 

However,  in  replacing  the  projective  invariant — the  Jacobian — by  a  certain  orthogonal  invariant,  a 
ool  can  be  obtained  which  is  sufficiently  strong  and  flexible  to  deal  with  these  problems  and  to  obtain 
theory  that  presents  m  many  respects  a  complete  analogy  with  the  theory  in  the  linear  case. 

2.2.  We  will  discuss  the  system  of  equations  ^ 

MX)  =  fM,  •  •  •  ,xn)  =  0         (/x=l,  .  .  .  ,n),  (1) 

vrhere  X  is  the  vector  with  the  real  components  Xi,  .  .  .  ,  x„  and  the  real  functions /;,  have  continuous 
irst  derivatives.     The  matrix  of  the  Jacobian 

^=|^'l  (2) 

\OXy\  ' 

—this  matrix  will  be  written  as 

—is  here  of  particular  importance.    If,  as  will  be  always  assumed  in  what  follows,  J 9^0,  we  will  write 
;he  inverse  matrix  to  {J)  as 

^x 
riie  symbol  ^~  denotes  here  the  corresponding  element  of  J~^.     Put  then 

A,(z)=|  sfl^YTj  (5) 


I2  measures  to  a  certain  extent  "the  degree  of  nonvanishing"  of  J.     Suppose  now  that  for  a  vector 
Xo{xl)  we  have 

f.{X,)  =  yl,  (6) 

vhcrc  the  vector  Fo(2/°),  that  is  its  norm,  is  "small".    Then  we  may  expect  that  in  the  neighbourhood 
)f  Xq  there  exists  a  solution  of  (1). 

This  statement  can  be  rigorously  proved  in  the  following  form: 

If  for  a  certain  D^O  in  the  neighbourhood  of  Xo 

\Xo-X\<D\Yo\  (7) 

he  relation 

A2(X)<D,  (8) 

iolds,  then  there  exists  a  solution  of  (1)  in  the  neighbourhood  (7). 

•  University  of  Basle,  Switzerland. 

•  In  what  follows  the  indices  /i,  v,  X,  c  will  always  run  from  1  to  n. 

29 

200431—53 3 


2.3.  The  theorem  of  section  2  can  be  proved  as  follows:  Consider  the  system  of  equations 

f,{x„  .  .  .,x„)-{l-t)f,ix'„  .  .  .,xl)  =  0  (9) 

where  t  varies  from  0  to  1.  For  sufficiently  small  values  of  t  there  exist  solutions  Xt  of  (9),  which  form 
a  continuous  arc  C,  beginning  in  Xo  and  contained  in  (7).  Then  we  have  for  the  length  of  arc  s(t)  along 
C,  measured  from  ^=0,  by  Cauchy-Schwarz  inequality 

(0=s(^'y=?(?  i  W^  ?  ?  (^y^ux.y=ux)^\  Y.\^     (10) 

0<^<A2(Z)|Fo|<^|ro|.  (11) 

Let  now  r,  0<t<1,  be  a  number  wdth  the  following  properties: 

A.  For  each  u,  0<'U<t,  there  exists  a  continuous  arc  C„  consisting  of  solution  points  P,(0<^<?/). 

B.  T  is  the  greatest  number  <1  with  the  property  A. 

It  follows  now  from  (11)  that  the  length  of  each  of  the  arcs  C„  is  <wZ)1Fo|<tZ>|Fo|,  so  that  all 
points  Pt  on  these  arcs  are  in  the  interior  of  the  neighborhood  (7).  There  exists  therefore  a  sequence 
ty,  0<f^<l,  such  that  t^  1  r,  and  that  the  points  P,^  converge  to  a  point  P*  also  situated  in  (7).  It 
follows  then  from  the  continuity  of /^  in  (7)  that  P*  is  a  solution  of  (9)  corresponding  to  t=T.  If  then 
T=l,  our  theorem  is  proved.     Suppose  now  t<^1. 

Then,  since  the  distance  lPoPiJ<T  i5|FoL  we  have  |PoP*|  <t  D  lFo|<C-C>|Fo|,  P*  lies  in  the  interior 
of  (7)  and  by  the  existence  theorem  there  exists,  for  a  sufficiently  small  e^O,  a  continuous  arc,  C*, 
through  P*  =  Pr  consisting  of  points  P,  with  \t—T\  <e.  But  then  it  follows  from  the  uniqueness  theorem 
that  a  point  P,^  lies  on  C*.  In  taking  then  the  portion  of  the  corresponding  arc  (?„  until  P,  and  then 
the  portion  of  C*  from  P,^  through  P*  and  beyond  P*,  we  see  that  t  is  not  the  greatest  number  <1 
with  the  property  A.     Therefore  we  have  t=1,  and  the  theorem  is  proved. 

2.4.  The  theorem  of  section  2  can  be  generalized  in  introducing  a  more  general  "norm"  in  the 
n-dimensional  vector  space.     Let  ^,  g  be  a  couple  of  numbers,  satisfying  the  relations 

-  +  -=1,  l<p<oo     1<g<oo.  (12) 

Then  we  define  as  the  ^-norm  of  the  vector  X(Xy) 

l^l.^[Z;l^.|^]^  (13) 

The  expression  (5)  can  be  generalized  to 

and  instead  of  the  theorem  of  section  2  we  obtain  the  theorem : 
If  for  a  certain  Z>>  0  in  the  neighbourhood  of  Xq  defined  by 

l^o-X|.<Z)lFo|„  (15) 

we  have  throughout 

Ap(X)<P>,  (KV) 

there  exists  in  this  neighbourhood  a  solution  of  (1). 

For  p  =  q=2  we  obtain  the  result  of  section  2.     The  two  other  particularly  important  cases  are 

^=00,    5=1;    p=l^    5=00. 

Observe  that  the  orthogonal  invariance  only  holds  for  p  =  q=2. 
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2.5.  In  order  to  apply  the  theorems  of  sections  2  and  4,  we  would  have  to  compute  Ap(X),  that  is 
iessentially  J~^,  throughout  a  whole  neighbourhood  of  Xq.  This  is  usually  not  feasible.  Practically, 
one  will  compute  Ap(Xo)  in  the  initial  point  Xq  and  will  try  to  obtain  an  estimate  for  the  variation' of 
Ap  in  a  neighbourhood  of  Xq.  This  can  be  done  systematically,  if /^  possess  finite  second  derivatives, 
in  using  the  following  important  inequality. 

Put 


Then  we  have 


grad 


d7. 


] 


1 

AoCX) 


<a;(X), 


(17) 


(18) 


.  where  grad  [1/A2(X)]  is  a  vectorwith  the  components  'd[ll^2{X)]l'dx^.     By  (18)  it  follows  from  the  relation 

1  1 


I   1        1   I    r^2 

|A2(Z2)        A2(Xi)|-jXi  ' 


These  inequalities  can  be  generalized  in  the  sense  of  section  4,  in  introducing 


We  obtain  then 
and  from  there  on 
where 


grad 


1 


1 


A.(Z)| 


x  dy^da-x 


<a:(X), 


< 


\X,-X,\,^\:(X''')dt, 


Ap(Z2)     Ap(XO 

X'''>=X,+t{X,-X,) 


(19) 
(20) 

(21) 

(22) 

(23) 
(24) 


However,  in  the  cases  p=\  or  g=l  it  may  become  necessary  to  write  the  above  inequalities  in  a 
more  special  form  in  which  the  one-sided  derivatives  are  used. 

The  inequalities  (18),  (22)  are  special  cases  of  very  general  inequalities  valid  for  general  determinants 
whose  elements  are  differentiable  functions  of  certain  variables. 

2.6.  The  inequalities  underlying  the  theorem  of  section  2  are  obtained  in  using  the  Cauchy-Schwarz 
inequality.  ]Much  better  results  can  be  obtained  in  using  an  orthogonal  invariant  of  more  involved 
kind. 

If  A={a^v)  is  an  nY^n  matrix,  an  7i-dimensional  vector  X  is  transformed  by  A  into  Y  where 


Y'=AX', 
and  there  exist  then  two  numbers  A>  X>  0,  such  that  for  any  choice  of  X  we  have 

XIX|<|F|<A|Z| 


(25) 


(26) 


and  that  here  A  cannot  be  replaced  by  smaller  numbers  and  X  by  greater  numbers.  It  is  well  loiown 
that  A  and  X  are  square  roots  of  the  greatest  and  least  fundamental  value  of  the  matrix  AA*.  We 
will  denote  these  expressions  by  A(^),  X(^).     If  |^|5^0,  we  have 


A(^-i)X(^)  =  l. 


(27) 


Now,  in  the  theorem  of  section  2,  ^2{X)  can  be  replaced  by  A{dxjdf^,)  and  since  this  expression 
never  exceeds,  and  is  usually  less  than,  A^iX),  we  obtain  in  this  way  closer  neighbourhoods  of  Xq. 
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The  computation  of  A(^)  and  X(^)  presents,  of  course,  even  much  greater  difficulties  than  that 
of  A2(X).  On  the  other  hand,  there  exist  in  this  case  uiequaUties  completel}^  analogous  to  (18).  These 
inequalities  can  be  formulated  in  the  following  way. 

Suppose  that  the  vector  A"  is  given  as  a  differentiable  function  of  a  real  variable  t  and  denote  by 
b  the  derivative  with  respect  to  t.     Then  we  have  i 


(28) 
(29) 


I 


These  inequalities  are  special  cases  of  two  inequalities  concerning  general  determinants  and  can  be 
generalized  even  further  in  introducing  the  notion  of  distance  in  a  Minkowski-space. 
2.7.     In  order  to  give  a  numerical  example  we  consider  the  system  of  equations 

^i=Zi-0.470  228  201  834  ^(Zj) -0.145  308  505  601  K{Zi)-Ci-^{), 

£•2  =  72  +  0.760  845  213  036  X(Zi)-0.615  536  707  435  K{Zz)-C2=Q, 

£'3  =  Z3  +  0.615  536  707  435  ^(Za) -0.760  845  213  036  K{Zi)-Cz^Q, 

£'4  =  Z4+0.145  308  505  601  K{Z,)+{)A1Q)  228  201  834  K{Zz)-Ci={), 


where 


72 


K{Z)=\g— ^;  (7i  =  5. 430  415  554  935;  (72=5.026  548  245  744: 

61  —  11  cos  Z  1        '^  > 

C3=4.345  243  969  113;  ^4=3. 769  911  184  308. 


Here  the  Jacobian  matrix  is 


d(Z.)' 


1  -0.4702  . 

0.7608  ..  .iiL'(ZO  1 

0  0.6155. 

.0.1453  ..  .2iC'(Zi)  0 


K'{Z,) 

0 

-0.1453  .  . 

.  K'{Z,T 

-0.6155  .  . 

•  K'{Z,) 

0 

K\Z,) 

1 

-0.7608  .  . 

.  K'{Z^ 

0.4702  .  . 

•  K'{Z,) 

1 

> 

Its  inverse  can  be  easih^  estimated  in  using  \K' {Z)\<  11/60. 
We  obtain  then  for  the  expressions  introduced  in  section  4 

IA2K2.O92;         |A„|<1.046;         |Ai|<4.183, 
independently  of  the  values  of  Z^.     The  approximate  values  of  Z  are  given  bv 

Z;=5.523  188;         Z^=4.847  788;         Z°=4.244  909;         Z2=3.684  244, 
and  the  corresponding  values  of  E  are 

£^=0.06207537;         £^=0.06975533;         £*=  0.06064 192;         £^=0.06289890. 
The  norms  of  the  corresponding  vectors  are  then 

|£°l2<10.407-10-^         I£°U<9.756•10-^•         |£°|i<15.372-10-' 
and  from  the  theorem  of  section  4  we  obtain  for  the  errors  of  the  approximations  (30) : 

|A"|2<21.772-10-^         |A"U<16.080-10-7;         |A°|j<40.810-10-7. 
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(30) 


If  we  compare  this  with  the  exact  solution  of  our  system: 

Zr  =  5.523  188  291  ..  .;        Zr  =  4.847  788  930  .  .  .;        73^  =  4.244  908  849  ..  .;        Z,"  =  3.684  244  294  ..  ., 

where  the  true  vahies  of  |A"jp  are 

|A"|2=10.29-10-";         |A"|„  =  9.3-10-';  |A"|l=16.66•10-^ 

|we  see  that  our  estimates  are  of  the  correct  order  of  magnitude. 

2.8.  The  expressions  introduced  in  the  above  discussion  of  the  system  (1)  and  the  properties  of 
these  expressions  permit  us  now  to  give  a  very  simple  proof  for  the  convergence  of  the  New^on-Kaphson 
method  and  to  deduce  very  close  estimates  for  the  error  implied. 

"\Vc  start  with  an  initial  point  Aq  and  define  an  infinite  sequence  of  points 

ij  AM''\--;(ii'^)         (k=O.L...)  (31) 

by  means  of  the  following  recurrent  procedure: 
Let 
11  Y.{y[^\...,yi^').        y';'=MA^)  (32) 

be  the  vector  formed  b}'  the  values  of  the  fy  in  ^4,.     Define  then  the  vector 

i  vM^\  .  .  .,  hi^^)  (33) 

from  the  system  of  linear  equations 

j:KV^4^^)  +  yi''  =  0         {v=l,  .  .  .,  n).  (34) 

Then  we  have 

^,+1=^,4-^,.  (35) 

I  In  order  to  ensure  the  possibility  of  this  procedure  being  continued  indefinitely,  we  make  the  follow- 

I  ing  assumptions:  For  a  certain  positive  number  M  and  a  couple  of  numbers ^J,  q  satisfying  (12)  we  suppose 
that  throughout  the  neighbourhood  U  of  ^4o,  defined  by 


we  have  for  p  —  q  =  2 
and  otherwise 

If  we  have 


{U)\X-Ao\,<2\r,o\p,  (36) 

M(X)<M        (X^U)  (37) 

[5j^g|7<M        iX.,...,X.^U).  (38) 

eo^2M\vo\p^p{Ao)<l,  (39) 


then  all  points  A,  defined  by  our  procedure  remain  inside  U  and  converge  to  a  point  Ain.  U  satisfying 
the  equations  (1).    We  have  then  in  particular 

and.  putting 

e,  =  2M\v,\pAp{A,):  (41) 

e.+,<2^;  (42) 

\Y.+^,<hM\v.\l.  (43) 
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The  estimates  obtained  can  be  sharpened  in  so  far  as  the  factor  2  in  (36)  and  (39)  can  be  replaced 
by  a  smaller  one,  if  2M\Yo\gAl(A^)<il.  These  estimates  correspond  exactly  to  those  obtained  by  the 
author  for  n=l.  For  n=2,  F.  A.  Willers  gave  in  1928  conditions  for  the  convergence  of  the  Newton- 
Raphson  procedm-e.  However,  these  conditions  involved  the  third  derivatives  of  /^.  In  1936  the  author  - 
gave,  again  for  n  =  2,  conditions  for  the  convergence  implying  only  the  second  derivatives  and  these 
conditions  were  then  generalized  to  the  general  n  in  the  Braunschweig  Thesis  of  K.  Bussmann.  However, 
these  conditions  gave  only  rough  estimates,  since  the  expressions  Ap,  Ag  were  not  used. 

On  the  other  hand,  a  part  of  our  results  of  section  8  has  been  given  and  considerably  generalized 
in  [4,  5,  6],  but  I  have  not  been  as  yet  able  to  study  these  papers. 

It  may  finally  be  mentioned  that  a  part  of  our  results  was  published  in  [7,  8,  9],  and  an  exposition 
of  the  bulk  of  theory  will  be  given  in  a  monograph  that  is  to  appear  in  the  series  of  Cahiers  Scientifiques 
(Gauthiers  Villars,  Paris). 
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3.  The  Geometry  of  Some  Iterative  Methods  ol  Solving  Linear 

Systems 

Alston  S.  Hoviseholder  ^ 

The  piu'pose  of  this  paper  is  to  give  a  simple  geometric  derivation  of  some  standard  iterative  methods 
for  solving  linear  systems,  and  to  obtain  these  as  particular  cases  of  a  fairly  general  family  of  methods. 
By  iterative  I  mean  infinitely  iterative,  and  hence  belonging  to  Forsythe's  class  III.^  My  pm-pose  is 
essentially  the  reverse  of  that  of  Dr.  Forsythe,  since  I  am  trying  to  find  the  common  identity  rather  than 
the  points  of  distinction.  The  point  of  \dew  taken  here  is  very  similar  to  that  fomid  in  a  set  of  notes,  not 
signed,  which  were  distributed  by  the  NBS  Institute  for  Numerical  Analysis  about  two  years  ago.  I 
hope  the  differences  are  sufficient  to  justify  this  exposition. 

In  order  that  each  symbol  may  be  easily  recognized  for  what  it  represents,  I  shall  use  always  small 
Greek  letters  for  scalars;  small  roman  letters  for  colimni  vectors;  and  capitals  for  matrices.  The  space 
of  a  matrix  will  be  the  space  of  its  coliunn  vectors.  When  we  happen  to  be  interested  in  the  rows,  the 
matrix  will  be  designated  as  a  transpose.  A  matrix  A  will  have  colmnns  a„  and  this  alphabetic  associa- 
tion will  be  followed  in  general,  except  that  the  identity  matrix  /  will  have  columns  et. 

In  a  space  whose  metric  is  represented  by  the  positive  definite  matrix  G,  a  projector  P  is  a  symmetric 
matrix  of  the  form 

with  the  understanding,  of  course,  that  the  coluimis  of  U  are  Imearly  independent.  If  the  columns  of 
U  are  taken  to  be  the  contravariant  representations  of  geometric  vectors  in  some  subspace,  and  if  r  is 
the  covariant  representation  of  any  vector,  then  Pr  is  a  linear  combination  of  columns  of  U  and  hence 
the  contravariant  represeutation  of  a  vector  in  the  subspace  of  U.  The  columns  oi  GU  are  the  covariant 
representations  of  the  same  vectors  of  which  the  columns  of  U  are  the  contravariant  representations. 
Since  PGU=  U,  it  follows  that  P  projects  any  vector  in  the  space  of  U  into  itself.  Since,  further, 
PGP=P,  P(I—GP)=0,  it  follows  that  for  any  covariant  r,  Pr  is  the  contravariant,  GPr  the  covariant 
representation  of  the  projection,  (I~GP)r  the  covariant  representation  of  the  residual,  and  the  projection 
and  residual  are  orthogonal.  Now  suppose  the  equations  to  be  solved  are  Ax—y,  where  A  is  nonsingular. 
For  the  iterative  procedure  one  selects  a  rule  for  obtaining  a  sequence  of  spaces  represented  by  the  matrices 
Uo,  Ui,  U2,  ■  '  ■',  then  if  Xa  is  any  approximation  to  the  solution  x,  one  projects  the  residual,  Sa=x—Xa 
orthogonally  upon  the  space  of  Ua,  adjoining  the  projection  to  Xa  to  obtain  Xa+i.  ^Miether  the  matrices 
Ua  are  known  explicitly  in  advance,  or  whether  each  is  selected  only  at  the  time  it  is  to  be  used  in  accord- 
ance with  some  criterion  that  cannot  be  applied  in  advance,  makes  very  little  difi'erence  in  principle. 
Clearly  the  norm  of  Xa+i  cannot  exceed  that  of  Xa.  It  seems  clear  intuitively  how  convergence  can  be 
assured,  and  in  any  case  I  shall  not  stop  for  convergence  theorems  here. 

If  the  matrix  A  is  positive  definite  it  is  natural  to  let  it  provide  the  metric  for  the  space.  The 
equations  then  define  the  contravariant  representation  x  of  the  vector  whose  covariant  representation 
is  the  known  vector  y.  Any  approximation  Xa  has  associated  with  it  a  covariant  vector,  ya=Axa,  which 
deviates  by  a  calculable  amount,  ra=y—ya,  from  the  required  one.  The  vector  r^,  which  is  computable, 
is  the  covariant  representation  of  the  deviation,  while  ««  is  the  (unknown)  contravariant  representation. 

"\Mien  we  associate  a  matrix  Ua  of  contravariant  vectors,  we  can  define  a  projector 

Pa=UaiU:AUa)-'U:, 
and  then  PaVa  is  the  contravariant  representation  of  the  projection  which  is  to  be  adjoined  to  Xa'. 

Xa-\-l  —  Xa\' L   a'a- 

'  Oak  Rldgc  National  Laboratory. 

'  Q.  E.  Forsythe,  Tentative  classification  of  inetbods  and  bibliography  on  solving  systems  of  linear  equations.    See  paper  1,  page  1. 
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Each  projection  involves  the  inversion  (UlAUa)~'  of  a  matrix  whose  order  is  equal  to  the  dimen- 
sionality of  the  space  Ua  upon  which  the  projection  is  to  be  made.  As  the  whole  purpose  of  the  method 
is  to  avoid  an  explicit  Inversion,  one  ordinarily  takes  f7„  to  be  a  single  vector  m„  so  as  to  have  only  a 
scalar  to  invert.  In  particular  instances,  however,  the  matrix  A  might  have  principal  minors  of  low- 
order  which  are  easily  ui verted.  If  so,  a  matrix  Ua  (or  each  of  an  infinite  subset  of  them)  can  be  made 
up  of  columns  e^  of  the  identity  so  that  the  product  U^AU  will  be  such  a  principal  minor.  In  general, 
of  course,  the  higher  the  dimensionality  of  the  space  upon  which  the  projection  is  made,  the  larger  the 
projection  and  hence  the  greater  the  gain  by  that  particular  step.  It  is  naturally  possible  to  choose 
other  matrices  Ua  yielding  matrices  of  low  order  that  are  easily  inverted,  but  hardly  worth  while  to 
spend  much  time  looking  for  them. 

Another  case  where  it  might  be  advantageous  to  use  matrices  Ua  with  more  than  one  column  arises 
when  a  special  purpose  machine  is  available  for  inverting  matrices  of  limited  size.  The  Oak  Ridge 
linear  equation  solver  will  solve  systems  of  order  300.  No  larger  systems  have  been  attempted  so  far, 
but  it  is  possible,  in  principle,  to  solve,  say,  a  system  of  order  600  by  inverting  two  matrices  of  order 
300  each  in  order  to  project  the  residuals  alternately  upon  two  300-dimensional  subspaces. 

The  simplest,  and  most  common,  procedure  is  to  take  each  Ua  to  be  a  single  vector  Ua-  The  method 
of  the  steepest  descent  takes  Ua—Va.  That  is,  one  would  like  to  project  upon  the  residual  itself,  in  which 
case  the  problem  would  be  completely  solved  in  one  step.  But  this  requires  the  contravariaat  repre- 
sentation Sa,  which  we  do  not  have.  So  we  take  /-„,  the  covariant  representation,  and  project  upon  the 
vector  for  which  this  is  the  confravariant  representation.  If  r^  happened  to  be  a  proper  vector  of  A, 
this  would  still  bring  us  out  at  one  step,  but  unfortunately  it  generally  is  not. 

The  more  common  choice  is  to  make  the  Ua  equal  to  the  ei  taken  in  some  sequence.  If  taken  in 
strict  cyclic  order  we  have  the  method  of  Seidel,  the  only  feasible  method  for  machine  computation. 
For  hand  work,  one  can  examine  Va  each  time  and  select  that  Ci  upon  which  Va  has  the  largest  projection 
as  in  the  method  of  relaxation.  If  pai  is  the  iih.  element  in  Va,  then  the  vector  etpai/au  is  the  projection 
upon  Ci,  and  as  this  is  the  contravariant  representation  the  squared  length  is  a:i,(Pai/an)^  =  Pa?/«ii.  The 
optimal  choice  of  Ci  is  determined  by  the  maximum  quotient  of  this  type. 

If  the  matrix  A  is  not  positive  definite,  nor,  perhaps,  even  symmetric,  the  equations  may  be  regarded 
in  either  of  two  ways,  as  representing  a  system  of  hyperplanes,  or  as  defining  the  resolution  of  a  known 
vector  y  along  the  column  vectors  of  A.  In  the  latter  interpretation  the  elements  of  x  are  the  required 
multipliers. 

In  this  case  it  is  natural  to  take  the  ordinary  metric,  (t=/,  in  which  case  the  projectors  are 

^    a         '^   ay-    a^   a/        '-^  a> 

and  the  projection  of  r«  upon  Ua  is 

Para=Ua(UlUa)-'Ulra. 

However,  the  projection  PaVa  is  to  be  adjomed  to  ya=Axa,  whereas  x„  itself  is  to  be  corrected  by  the 
vector  A-^PaVa.     Therefore,  to  make  this  feasible  we  must  set  Ua=AVa  and  write 

Pa  =  AVaiV:A^AVay'V:Ar 

Hence 

.u+:=.ra'{-Va(V:A'AVay'V:A^ra. 

One  chooses,  thus,  the  F„,  and  only  indirectly  the  Ua- 

The  appearance  of  the  product  A'^A  relates  this  method  to  the  result  of  replacing  the  equations  by 
the  equivalent  set  A^Ax^A^y  and  proceeding  as  before  with  a  matrix  that  is  now  positive  definite. 
However,  the  complete  product  A^A  is  not  required,  but  only  products  of  the  type  {AVaY{AVa).  In 
particular,  if  each  r„  is  a  single  Ci,  then  one  requires  only  the  afa^  to  be  inverted.  These  can  be  com- 
puted in  advance  and  used  in  rotation  to  give 

Xa+x=Xa^-ei{aJai)-HaJra). 
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A.ny  selection  of  the  et  as  in  the  method  of  relaxation  is  ruled  out  since  to  make  a  selection  one  would  have 
:o  compute  every  afva  and  compare  magnitudes,  whereas  having  computed  one  of  these  scalar  products 
oaost  of  the  work  has  been  done  for  that  particular  projection.  Note  that  since  the  projection  is  made 
apon  €i  only  a  single  element  of  Xa  is  changed,  but  that  no  single  element  of  the  residual  r^+i  is  necessarily 
eliminated. 

As  was  indicated  m  the  INA  notes,  this  method  applies  directly  to  the  equations  of  least  squares. 
These  equations  have  the  more  general  form  Ax  =  y-{'e,  where  e  is  required  to  be  orthogonal  to  the  space 
of  A.  The  usual  procedure  is  to  multiply  by  A'^,  which  eliminates  e  and  yields  a  system  A'^Ax=A'^y 
with  positive  definite  matrLx.  However  if  one  projects  upon  subspaces  represented  by  Ua=AVa,  then 
one  is  projecting  upon  subspaces  of  the  space  of  .1,  and  the  residual  is  always  orthogonal  to  A.  It  is 
clear  intuitively  that  except  for  very  special  choices  of  the  matrices  Va  the  sequence  ya  should  approach 
:a  limit,  which  will  necessarily  lie  in  the  space  of  ^1,  and  that  the  residuals  ?/— ?/„  will  approach  a  vector 
orthogonal  to  this  space.     However,  again,  I  shall  not  stop  to  give  a  proof. 

To  consider  the  hyperplane  interpretation  let  the  equations  be  written  in  the  form  A^x=z.  Then 
each  column  at  of  A  represents  the  normal  to  one  of  the  hyperplanes.  The  projection  of  the  deviation 
\Sa  upon  Ua  is 

'but  since  it  is 

Vet ' —  ^        2  a  —  -^    ^a 

that  is  known,  and  not  §«  itself,  we  must,  as  before,  let  Ua=AVa  so  that 

P^s„=AV.iVlA^AV^)-'V:t^, 
,and 
i  x„+,=x^+AV^{VlA''AV„)-'VX. 

If  for  Fa  one  takes  a  vector  ei,  the  result  is 

Xa+l=Xa  +  ai{aJai)~'^Tai 


'  Again  one  needs  then  only  the  diagonal  elements  of  A'^A,  and  not  the  entire  matrix.  To  follow  the 
method  of  relaxation,  one  should  choose  that  i  for  which  the  projection  is  maximized,  and  the  square  of 
the  projection  is  TaJiafai).     Whatever  i  is  chosen,  the  projection  is  along  the  vector  a<  so  that  in  general 

r  every  element  of  Xa  is  affected.  However,  since  the  new  residual,  x—Xa+i,  is  orthogonal  to  at,  the  vector 
Xa+i  taken  from  the  orighi  terminates  on  the  ith  hyperplane,  so  that  by  the  projection  the  ith  residual 
element  is  caused  to  vanish.  This  is  to  be  contrasted  with  the  former  case  where  no  residual  element 
will  necessarily  vanish  but  onlj^  one  element  of  Xa  requires  modification. 

The  method  of  steepest  descent  takes  for  F„  the  single  vector  t^.     The  reason  is  that  if  one  defines 
the  function 

^(x)  =  {z'^-x'^A)  {z-A'^x)=z'^z-2x'^Az^x'^AA'^x, 

then  ip{x)  takes  on  the  minimum  value  zero  at  the  point  x  which  satisfies  the  equations,  and  has  the  gradient 
—  2  At  a  at  the  point  Xa.  Hence  taknig  /<,  for  Va  is  equivalent  to  takmg  .1  ta  for  Ua,  that  is  a  vector  in  the 
direction  of  the  gradient,  in  which  direction  the  variation  of  the  function  is  most  rapid.  Equivalent 
to  the  original  set  of  equations  is  the  system  AA'''x=y,  where  y=Az.  The  gradient  of  tp{x)  at  Xa  is  also 
expressible  as  —2ra,  where  ra  =  y—AA'^Xa,  which  accounts  for  the  choice  of  vector  upon  which  one  projects 
in  applying  the  method  of  steepest  descent  in  the  ordinary  case  where  the  matrix  is  positive  definite. 


2094.^1 — 53- 


37 


4.  Solutions  of  Linear  Systems  of  Equations 
on  a  Relay  Machine 

Carl-Erik  Frbberg  ^ 

Characteristics  of  the  Bark 

The  BARK  has  been  described  in  some  detail  elsewhere,^  but  for  concreteness  a  brief  review  will  be 
given  here.  Numbers  are  represented  in  the  form  ±2*''-g,  where  p  is  a  six-digit  binary  number,  and 
q  is  a  24-digit  binary  number.  Transfer  within  the  machine  is  accomplished  along  three  busses,  each 
of  which  contains  32  channels.  In  its  first  stage  of  development,  the  machine  had  50  relay  registers, 
100  constant  registers,  and  840  order  points;  by  now  the  corresponding  figures  are  100,  200,  and  1,200. 
The  arithmetic  circuits  consisting  of  an  adder  and  a  multiplier,  carry  out  transfer,  addition,  and  multipli- 
cation, and  division  is  accomplished  through  an  iteration  process.  Further,  there  are  special  circuits 
that  can  handle  other  elementary  operations  (for  example,  transfer  of  exponents,  of  the  numerical  part 
only,  of  the  fractional  part  of  a  number,  and  right  or  left  shift).     Extraction  of  the  integral  part  of  a 

:number  needs  no  special  order,  as  this  can  be  done  by  adding  an  "integral  zero",  that  is,  zero  with  the 
exponent  +24. 

Orders  are  written  in  the  form:  n  A  op  signs  BCD,  where  A  and  B  are  the  addresses  of  the  two 
numbers  which  shall  be  combined  by  "op";  C  is  the  address  where  the  result  should  be  stored  and  D  the 
next  order  to  be  executed.  Normally  the  order  n  is  followed  by  n+ 1 ;  unconditional  jumps  are  made  by 
means  of  plugged  connections  on  the  sequence  panel,  while  conditional  jumps  also  need  selectors,  con- 

•  sisting  of  relay  pyramid  circuits. 

I  Input  and  output  devices  make  use  of  standard  teletype  equipment.  There  are  five  tape  readers, 
five  tape  punches,  and  one  page  printer  available.     This  equipment  is  also  used  as  an  extra  external 

.memory,  although  with  limited  fiexibility. 
The  operation- times  are  approximately: 

Transfer 100  ms. 

Addition 150  ms. 

Multiplication 250  ms. 

Methods  for  Solution  of  Linear  Systems 

There  exist  several  standard  methods  for  solution  of  linear  systems:  direct,  iterative,  and  statistical 
methods.  Out  of  these,  only  direct  methods  have  been  used  on  the  bark,  and  some  of  them  will  be 
discussed  briefly. 

For  concreteness,  we  introduce  the  equation    . 

Ax^b,  (1) 

where  /I  is  a  square  matrix  of  order  n  and  x  and  b  column  vectors.  Now  it  turns  out  that  the  amount 
of  work  needed  for  evaluation  of  the  inverse  matrix  is  onl}^  a  factor  2  to  3  times  that  for  solution  of  the 
equation  for  one  special  column  vector  b.  Because  of  the  great  advantage  in  knowing  the  inverse,  for 
example,  for  the  solution  of  the  equation  for  any  vector  b  or  for  the  construction  of  more  accurate  solu- 
tions from  a  first  approximation,  one  very  often  does  this  extra  amount  of  work. 

The  best-known  direct  methods  are  those  of  Gauss,  Jordan,  and  Cholesky,  but  there  are  several 
other  methods  that  can  also  be  used.^-*     The  method  of  Gauss  consists  in  forming  an  upper  triangular 

'  University  of  Lund,  Sweden. 

'  Kjellberp-Xeovius,  The  bark,  A  Swedish  general-purpose  relay  computer,  MTAC  [33]  V,  29  (1951). 

»  L.  Fox,  TI.  D.  Uuskey,  and  J.  H.  Wilkinson,  Notes  on  the  solution  of  algebraic  linear  simultaneous  equations,  Quart.  J.  Mech.  and  Applied  Math.  1, 
149  (1948). 

*  A.  M.  Turing,  Kounding-ofl  errors  in  matrix  processes,  Quart.  J.  Mech.  and  Applied  Math.  1,  287  (1948). 
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matrix  C  by  left-multiplication  by  a  sequence  of  matrices  Ji  .   .  .  Jn-i,  where  the  matrix  J^  differs  from  : 
the  unit  matrix  only  in  the  ^th  column.     Then  we  have 

(?=  J,_,  .  .  .  J,A,  (2) 

and  the  equation  has  taken  the  form: 

The  unknown  quantities  x  can  now  easily  be  obtained  one  after  another  by  a  simple  substitution  process. 
The  inversion  of  C  proceeds  analogously,  and  then  the  inverse  of  .4  can  be  found  directly: 

A-'  =  C-KIn-i  .  .  .  Ji.  (4) 

The  method  of  Jordan  is  rather  similar,  but  here,  instead  of  the  triangular  matrix,  a  diagonal  matrix 
is  formed  by  elimination.  The  Cholesky  method  is  originally  applicable  to  symmetric  matrices  only 
and  consists  in  expressing  the  matrix  A  in  the  form 

A=LU,  (5) 

where  Z  is  a  lower  triangular  matrix  and  L'  its  transpose.  Now  it  can  be  proved  that  A  being  a  non- 
symmetric  matrix,  the  principal  minors  of  which  are  nonsingular,  there  exists  a  unique  resolution  ' 

A  =  LDU.  (6) 

where  L  and  U  are  unit  lower  and  unit  upper  triangular  matrices  and  D  a  diagonal  matrix.  In  all  these 
cases,  the  inversion  of  a  matrix  is  reduced  to  inversion  of  a  triangular  matrix,  which  is  performed  by 
iterated  substitutions. 

The  Automatic  Computation  Aspect 

There  are  certain  limitations  in  the  choice  of  computational  method.    Due  to  the  small  capacity  of 
the  internal  memory  in  a  relay  machine,  it  is  generally  impossible  to  store  the  whole  matrLx  there.  'A 
Instead,  it  is  put  on  tape,  row  by  row,  or  column  by  column.    Because  punching  is  rather  time-consuming,  ,| 
it  is  desirable  that  storing  of  inteiTnediate  results  can  take  place  in  the  internal  memory,  which  means  )j 
that  only  one  single  row  or  column  can  be  handled  simultaneously.    All  elimination  methods  being  linear, 
it  is  evident  that  the  computations  can  be  checked  by  taking  the  sum  of  all  elements  and  performing 
the  same  operations  on  it.    This  procedure  takes  very  little  time  and  only  a  few  extra  orders. 

The  operation  times  on  a  relay  machine  being  rather  long,  it  is  desirable  to  choose  a  method  where 
the  total  amount  of  operations  is  as  small  as  possible.  For  such  a  method  the  number  of  orders  will  be 
larger  than  for  other  methods,  but  this  only  means  a  little  extra  work  in  coding  the  problem.  On  the 
other  hand,  by  using  a  fast  electronic  computer,  one  can  afford  to  use  more  operations,  that  is,  longer 
time,  in  order  to  simplify  the  programming.  For  this  reason  the  direct  methods  seem  to  be  best  adapted 
to  a  relay  machine,  while  iterative  or  statistical  methods  might  be  more  suitable  for  a  fast  computing 
machine. 

On  the  BARK  several  linear  systems  (origin:  surveying)  have  been  solved  by  Gauss'  and  Jordan's 
methods.  The  method  of  Cholesky  offers  no  special  advantages  here.  It  turns  out  that  the  time  con- 
sumed when  using  the  Gauss  method  for  inversion  of  an  unsymmetrical  matrix  is  only  75  to  90  percent 
of  that  required  when  using  the  Jordan  method.  On  the  other  hand,  the  Gauss  method  has  the  dis- 
advantage that  the  inversion  must  take  place  in  three  steps:  (a)  Forming  the  triangular  matrix  T,  (2); 
(b)  inversion  of  C;  (c)  forming  A-^,  (4);  whereas  the  Jordan  method  w^orks  in  two  steps.  Thus  it  is 
obvious  that  the  Gauss  method  takes  more  memory  space  and  more  orders  (116  compared  with  87) 

'  J.  von  Neumann  and  II.  H.  Goldstine,  Numerical  inverting  of  matrices  of  high  order,  Bui.  Am.  Math,  ?oc.  53.  1021  (1947). 
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than  the  Jordan  method.    So  we  can  make  the  conclusion  that,  when  enough  memory  space  is  available 
(the  order  of  the  matrix  being  not  too  large),  the  Gauss  method  should  be  preferred  to  the  Jordan  method. 


I,  Lastly,  it  should  be  mentioned  that  there  exist  some  very  special  cases  when  the  solution  ought  to  be 
made  b}-  hand,  namely,  (a)  when  there  are  onl}'  very  few  elements  outside  the  diagonal  in  A;  and  (b) 
when  just  two  to  three  correct  figures  are  required. 

I 

Results 

As  mentioned  above,  solution  of  linear  systems  of  equations  has  been  accomplished  with  the  Gauss 
and  Jordan  elimination  methods.  Matrices  of  orders  n  =  8,  14,  20,  and  28  have  been  treated.  Inversion 
of  matrices  of  orders  n  =  8  and  20  has  been  perfomied  with  Jordan's  method.*  The  coding  of  the  problems 
is  straightforward;  because  of  the  floating  binary  point,  positioning  for  size  is  unnecessary.  The  matrix 
elements  are  translated  to  octal  sj'stem  and  put  on  the  tape;  conversion  from  octal  to  binary  form  is 
trivial. 

The  time  consumed  appears  in  the  following  table  (hours) : 


n 

Solution 

Inversion 

Unsymmetrical 
(Jordan) 

Symmetrical 

(Gauss) 

Unsymmetrical 
(Jordan) 

8 
20 

0.  6 
3.5 

0.4 
2.  1 

1.  1 
7.5 

These  results  w^ere  obtamed  when  the  memory  capacity  was  only  50  places;  by  now  100  memory  places 
being  avaOable  the  times  can  be  reduced  considerably.  No  definite  figures  have  been  obtained  so  far, 
jbut  the  following  formulas  will  give  an  idea  of  the  time-consumption  expected  (in  seconds). 

Solution : 


Inversion: 


T=0.8n»  +  2.0n^ 

Unsymm. 

Jordan. 

r=0.6n'4-2.0n2 

Symm. 

Jordan. 

r=0.5n»  +  2.7n2 

Unsymm. 

Gauss. 

T=0.2n^  +  2.7n? 

Symm. 

Gauss. 

T=1.9n='  +  3.9ri» 

Unsymm. 

Jordan. 

T=1.2n?  +  3.9n' 

Symm. 

Jordan. 

r=1.4n3  +  5.6n« 

Unsymm. 

Gauss. 

T=0.7v?  +  5.&n^ 

Symm. 

Gauss. 

For  example,  the  time  for  inversion  of  a  20X20  unsymmetrical  matrix  will  go  down  from  7.5  to  4.6 
hours  with  the  Jordan  method  and  to  3.7  hours  with  the  Gauss  method. 

Most  of  the  matrices  treated  have  been  "well-conditioned",^  and  in  these  cases  the  results  have 


I       •  All  these  computations  were  performed  under  the  direction  of  O.  Karlqvist. 
'  Sec  footnote  4. 
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been  correct  to  about  six  digits  (the  machine  itself  works  with  slightly  more  than  seven  decimal  digits) 
As  an  example,  the  following  check  is  given: 


-3 


10^  {AA-'- 

-E) 

(n=8) 

+  2 

0 

-1 

0 

-1 

0 

0 

-5 

+1 

0 

0 

0 

-1 

0 

0 

-3 

0 

-6 

-3 

0 

+  1 

+  2 

-4 

+  3 

+1 

-1 

+  3 
-9 

+  2 
-2 

-7 

+1 
-1 

-3 
-3 

where  yl  is  a  symmetric  8X8  matrix  with  a  clear  diagonal  dominance.  The  ''condition-nmnber"  of  A 
(defined  as  \ln'N{A)-N{A-^))  was  5.5. 

On  the  other  hand  also,  some  rather  ill-conditioned  systems  have  been  treated;  in  these  cases  the 
accuracy  decreased  to  about  three  correct  (decimal)  figures. 

Nevertheless,  these  results  seem  to  indicate  that  the  figures  given  by  v.  Neumann-Goldstine  *  nor- 
mally are  too  pessimistic,  and  they  rather  point  in  favor  of  their  new  results.' 

•  See  footnote  5. 

»  John  V.  Neumann  and  E.  H.  Qoldstine,  Numerical  inverting  of  matrices  of  high  order,  II.    Proo.  Am.  Math.  Soc.  S,  18S  (1951). 
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5.  Some  Special  Methods  of  Relaxation  Technique 


Eduard  Stiefel  ^ 


5.1.     The  principle  of  relaxation  methods  is  to  replace  the  direct  solution  of  a  symmetrical  system 
of  linear  equations 


SlciikUk+li  =  0,         aa=aki,         i=l,  2,  .  .  .,  71 


fc=i 


by  the  problem  of  minimizing  the  quadratic  fujiction 


^   i,k  i 


It  will  be  assumed  that  the  quadratic  form 


^aikUiUk 


(1) 


(2) 


(3) 


is  positive  definite.     Introducing  the  n-dimensional  vectors 

U=(Ui,V2,     .     .     .,    Un),  l=^(li,h,     .     .     .,    In), 

I 
I 

j   which  may  be  considered  as  points  in  a  space  i?",  and  also  the  matrix  operator  Z>  =(««),  the  system  of 
i   equations  may  be  written  as 

I  Du-\-l=0  (4) 

and  the  quadratic  function  as 


F{u)=-  uDu+l'U. 


(5) 


STEEPEST  DESCENT 
F:CONSTANT 


Figure  5.1. 


>  Professor  of  Applied  Mathematics,  Swiss  Federal  Institute  of  Technolouy,  Zurich,  Svvitveiland. 
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If  the  exact  solution  u  be  replaced  hy  a  trial  solution  v={v,,  ih,  .  .  .,  Vn),  then  (4)  must  be  changed  to 

Dv+l  =  r.  (6) 

r=(r^,  rj,  .  .  .,r„)  is  called  the  residue  vector,  and  its  components  are  the  residues.     Taking  the  cjuad- 
ratic  function  F{v)  of  the  trial  vector  v,  we  note  that 

r=gr&dF(v).  (7) 

The  idea  now  is  to  make  the  residues  successively  smaller,  or,  in  other  words,  to  minimize  F(u).  The 
step-by-step  procedure  by  which  this  is  done  may  be  best  understood  by  looking  at  figure  5.1.  It  shows 
a  two-dimensional  cross  section  through  i?",  which  has  been  passed  through  the  trial  point  v  and  the  cor- 
responding residue  vector  r.  The  intersections  of  the  surfaces  F=  constant  with  this  cross  section  are 
a  set  of  concentric  elhpses  (contour  lines).  As  a  consequence  of  (7),  the  residue  vector  r  is  normal  to 
the  ellipse  passing  through  v.  One  step  in  the  computation  now  consists  m  choosing  a  suitable  direction 
vector  p  and  moving  out  from  the  trial  point  in  the  direction  of  p  until  F(v)  reaches  a  minimum  value. 
This  will  be  the  case  at  some  point  u'  at  which  the  vector  p  is  tangent  to  one  of  the  contour  Imes.  v'  is 
our  new  trial  point,  and  the  new  residue  vector  r'  at  this  point  will  have  to  be  perpendicular  to  p. 
Analytically  we  have 

v'  =  v+\p,  (8) 

where  X  is  a  still  undetermined  scalar.     From  (6)  it  follows  that 

r=Dv+l 

r'-r=D{v'-v)  =  \Dp 
or 

r'  =  r+\Dp.  (9) 

Writing  the  scalar  product  of  two  vectors  as  a-b,  the  condition  that  r'  be  orthogonal  to  p  becomes 
r''p=0,  and  hence 

{r+\Dp)p=rp+\Dpp=0 
or 

(10) 

After  determining  X  from  this  formula,  the  new  trial  point  is  given  by  (8)  and  the  new  residue  by  (9) . 

5.2.  The  various  relaxation  methods  differ  from  each  other  in  the  way  of  choosing  the  direction 
vector  p.  The  simplest  procedure  is  to  take  p  parallel  to  one  of  the  coordinate  axes.  This  is  essentially 
what  is  done  also  in  Southwell's^  "block"  and  "group"  relaxations,  which  have  been  studied  in  this 
connection  by  the  author.^  This  procedure  has  the  advantage,  that  the  computer  is  left  considerable 
freedom  in  choosing  the  direction  vector  to  suit  the  residue  situation.  On  the  other  hand,  with  the  use 
of  automatic  calculators  it  is  necessary  for  p  to  be  determined  by  some  definite  rule.  This  is  most 
commonly  done  by  using  the  method  of  steepest  descent,  in  which  p  is  chosen  as  p=  —  r;  that  is,  we 
look  for  the  minimum  along  the  normal  to  the  contour  liae  at  the  point  v.  From  our  general  formulas 
we  have 

v'=v—\r,     r'  =  r-\Dr  (11) 

with 

'*''*  (12) 


»  R.  V.  Southwell,  Relaxation  methods  in  engineering  science  (Oxford,  1946);  Relaxation  methods  in  theoretical  physics  (Oxford,  1946). 
3  E.  Stiefcl,  Ueber  einlge  Methoden  der  Relaxationsrechnung,  Z.  angew.  Math.  Phys.  (Jan.  1952) 
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Figure  5.2. 

Thus,  in  this  case  X  is  the  reciprocal  Rayleigh  quotient  of  r.  Occasionally,  a  computation  is  made, 
using  formulas  (11)  but  assuming  X  to  be  constant  instead  of  calculating  it  out  at  each  step,  the  so- 
called  weak  method  of  steepest  descent.  A  special  case  of  this  procedure  is  Liebmann's  method  for 
the  numerical  solution  of  Laplace's  boundary  value  problem. 

5.3.  A  still  better  choice  of  p  can  be  made  by  du'ecting  it  toward  the  center  of  the  ellipses.  Figure 
5.2  shows  the  situation  in  greater  detail.  Let  Vk  be  the  Hh  trial  point,  which  was  determined  with  the 
help  of  the  direction  vector  Pk-i,  and  let  the  corresponding  residue  vector  be  r*.  On  passing  a  two- 
dimensional  plane  through  Uk  and  r^  (the  plane  of  fig.  5.2),  the  surfaces  F=constant  again  give  a  set  of 
contour  lines.  As  we  saw  in  the  first  paragraph,  it  is  characteristic  of  a  step  in  the  relaxation  that  the 
ellipse  passing  through  Vk  is  tangent  to  Pk-i-  It  is  now  desired  that  the  next  direction  vector  pass  through 
the  center  of  the  ellipse;  that  is,  Pk  and  Pk-i  must  be  conjugate  directions  in  respect  to  the  ellipse,  or 

Pk-Dpk-,  =  0.  (13) 

If  we  put 

Pk=  —  rk  +  tk-\Pk-\,  (14) 

then 

—  Yk-Dpk-i-^  ek-\Pk-vDpk-x  =  0 
or 

rk-Dpk-i 


€fc-r 


(15) 


Pk-vDpk-i 

From  formulas  (8),  (9),  and  (10),  we  find  for  the  new  trial  point 

P*+i  =  «'*+XtP*     rt+,  =  rt  +  XfcDp* 

with 

Phlfpk 

In  order  to  start  the  calculation,  a  first  trial  point  vq  may  be  arbitrarily  chosen,  the  corresponding  residue 
vector  computed,  and  the  first  direction  vector  taken  as  po=  —  ''o-  ^^  interesting  property  oj  this  relaxation 
method  is  that  the  solution  of  the  n  equations  in  n  unknowns  will  be  found  in  precisely  n  steps.  In  other 
words,  we  have  not  only  a  method  of  successive  approximation,  but  also  an  algebraic  scheme  for  the  exact 
solution.^ 

•  Proof  of  this  proi)crty,  as  well  as  a  further  slrapllflcation  of  the  method,  may  be  found  in  the  artlcli;  cited  in  footnote  3, 
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Figure  5.3. 


Table  1. 


k 

Pk 

Dpk 

Tfc+l 

Pk 

Dpk 

rk+i 

Pk 

Dpk 

rk+i 

-10000 

0 

0 

0 

+  10000 

+  40000 

0 

0 

-10000 

-2500 

0 

0 

1 

+  1250 

0 

0 

+  2500 

+  8750 

0 

0 

-2500 

-714 

2 

+  306 

0 

0 

+  612 

-    1 

0 

+  714 

+  2244 

0 

3 

+   93 

0 

0 

+  186 

0 

0 

+  217 

0 

0 

0 

+   20 

0 

0 

+  40 

-    1 

0 

+  47 

+   1 

0 

2991 


982 


312 


0 

0 

0 

0 

0 

-10000 

-2500 

0 

0 

0 

0 

1 

+  2500 

+  8750 

0 

0 

-5000 

-1429 

0 

0 

0 

2 

+  612 

-    1 

0 

+  1429 

+  4492 

0 

0 

-2143 

-682 

3 

+  186 

0 

0 

+  434 

0 

0 

+  682 

+  2077 

0 

4 

+  40 

-    1 

0 

+  94 

+   2 

0 

+  147 

-   1 

0 

982 


625 


268 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

-2500 

-714 

0 

0 

0 

0 

0 

0 

2 

+  714 

+  2244 

0 

0 

-2143 

-682 

0 

0 

0 

3 

+  217 

0 

0 

+  682 

+  2077 

0 

0 

-1364 

-448 

4 

+  47 

+   1 

0 

+  147 

—   1 

0 

+448 

+  1498 

0 

312 


(ro,ro)  =  100-10« 
(ri,ri)-12.5.10« 
(r2,r2)  =  3061633 
ir3,n)=  930248 
{n,n)=^   200704 


268 

{po,Dpo)=  400-106 
(pi,Z)pi)=4375.10« 
(P2,Z)P2)=  9622276 
(p3,Dp3)=  2833028 
{pi,Dpi)=     671012 


134 


Xo  =  0.  25 
X,  =  0.  28571 
Xo  =  0.  31818 
X3  =  0.  32836 
X4  =  0.  29911 


«o=0.  125 
£1  =  0.24493 
€2  =  0.  30384 
€3  =  0.  21575 
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This  scheme  may  also  be  used  to  invert  the  matrix  (aa)  of  eq  (1).     The  elements  of  the  inverse 
matrix  are 


5a=z; 


PfiPpk 
p,Dp, 


where  ppi  is  the  ith.  component  of  the  pth  direction  vector.  Furthermore,  the  successive  residue  vectors 
To,  Ti,  r2,  .  .  .  form  an  orthogonal  system,  a  property  that  may  be  used  to  calculate  the  characteristic 
values  of  the  matrix  (a,*).     The  resulting  procedure  is  similar  to  that  suggested  by  Lanczos.^ 

Table  1  contains  a  simple  example,  a  numerical  solution  of  Laplace's  equation  with  the  boundary 
values  shown  in  figm-e  5.3.  The  function  u  is  to  be  determmed  for  the  nine  inner  points,  such  that 
Du=0,  where  D  is  the  operator  given  by  the  small  cross  (4,  —1,  —1,  —1,  —1).  Each  box  in  the  table 
corresponds  to  one  of  the  nine  points  and  contains  the  successive  values  of  pk,  Dp^,  and  rk+j.  The  pro- 
cedure was  started  with  ro  =  0.     The  values  of  the  desired  function  u  are  given  at  the  end. 

This  method  is  especially  suitable  when  Z>  is  a  difference  operator,  such  as  results  from  the  approxi- 
mation of  a  partial  differential  equation  by  a  difference  equation.  The  calculation  of  Dv  is  then  not  so 
long  as  it  would  be  if  D  were  a  more  general  matrix  (a,fc).  At  the  present  time,  this  method  is  being  used 
on  the  Zurich  Relay  Calculator  to  compute  the  elastic  deformations  of  a  plane  plate  under  29  different 
loading  conditions.  The  results  of  each  step  in  the  relaxation  are  punched  into  tape  and  fed  back  into 
.the  machine  again  for  the  next  step. 


^x 


Figure  5.4. 

5.4.  In  a  somewhat  different  manner  the  following  problem  has  been  handled  on  the  Zurich  ma- 
chine. The  Airy  stress  function  is  to  be  calculated  for  the  profile  of  a  dam  supported  elastically  by  the 
ground  (fig.  5.4).  The  ground  is  assumed  to  be  a  half-plane.  The  problem  yields  the  partial  differential 
equation  AAm=0  and  the  following  boundary  conditions  between  the  stress  functions  Ui  in  the  dam  and 
U2  in  the  ground: 

Ui=U2, 


dy      dij 
d^2_^  ^.il  ^1 

d^2_  J_  d^,_79    d^Ui       13 
d?/3~15  d?/3      45bx^diyi8' 


To  solve  the  problem,  the  triangle  ABC wns  covered  by  a  grid  containing  139  points.  The  difference 
operator  relating  the  value  of  the  function  at  each  grid  point  to  the  values  at  neighboring  points  is  shown 
in  figure  5.5.  In  the  ground  below  the  dam,  the  explicit  solution  developed  by  Boussinesq  for  the 
boundary  value  problem  AAi/  =  0  in  a  half-plane  was  used,  which  required  some  modification  of  the 

•  C.  Lanczos,  An  iteration  method  for  the  solution  of  the  eipenvaluc  problem  of  linear  difTerential  and  integral  operators,  J.  Research  N  BS  45.  255-282  (1950) 
RP2133. 


difference  operator  along  the  line  AB.  The  139  equations  in  139  unkno^vns  were  solved  by  usmg  a 
relaxation  method  related  to  the  one  in  paragraph  5.3,  in  which  an  approximate  computation  of  the 
first  eigenfunction  was  used  to  eliminate  the  contribution  of  that  function  to  the  residual  errors.  ,  The 
work  was  carried  out  on  the  machine  in  blocks  of  16  grid  points,  whose  residues  were  simultaneously 
reduced  to  zero.  Incidentally,  by  an  application  of  the  theory  of  group  characters,  these  16  equations 
could  be  reduced  to  two  sets  of  three  equations  each  plus  fom-  other  equations.  Each  such  block  took 
30  minutes  of  machine  time  and  the  entire  calculation  to  five  significant  figures  about  100  hours. 


r 

f ' 

-8              *Z 

Tl ^-8                      20           ■       U                   ' 
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-8 
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6.  Errors  of  Matrix  Computations 

Paul  S.  Dwyer^ 

Introduction 

Considerable  work  has  been  done  during  recent  years  on  the  general  question  of  the  accumulation 
'of  errors  when  sets  of  approximate  simultaneous  linear  equations  are  solved  b}-  approximate  methods. 
I  refer  particularly  to  the  work  of  Etherington  [1],^  Tuckerman  [2],  Lonseth  [3,  4],  Hotelling  [5,  6], 
Satterthwaite  [7],  von  Neumann  and  Goldstine  [8,  9],  Tm-ing  [10],  and  Ostrowski  [11],  as  well  as  to  the 
work  of  others  mentioned  in  the  list  of  references.  My  list  of  references  includes  about  30  titles  and  is 
not  as  inclusive  as  is  the  list  of  G.  E.  Forsythe  [12],  though  the  area  covered  in  my  list  corresponds  in  a 
general  way  to  that  of  his  last  main  subdivision  IV. 

It  is  my  purpose  here  to  present  a  discussion  of  the  nature  of  the  general  mathematical  problem 
referred  to  above,  together  with  indications  as  to  the  various  methods  proposed  for  handling  the  different 
phases  of  it.  Perhaps  I  should  add  that  this  paper  is  related  to  the  material  in  chapters  2  and  17  of  my 
book  on  Linear  Computations  [13],  although  the  presentation  here  is  more  general  since  it  is  not  directed 
specifically  toward  problems  involving  a  relatively  small  number  of  variables.  Problems  involving  few 
variables  can  be  handled  with  desk  machines  by  exact  pivotal  methods  or  by  approximate  pivotal  methods 
which  do  not  involve  extensive  accumulations  of  rounding  off  errors. 

The  problem  as  proposed  leads  to  an  examination  of  the  natm-e  of  the  fundamental  errors  involved. 
This  in  turn  demands  an  examination  of  the  numbers  used  to  express  these  errors.  The  question  of 
how  these  fundamental  numbers  can  be  combined  leads  us,  in  the  general  matrix  statement  of  the 
problem,  to  an  examination  of  the  errors  of  addition,  subtraction,  and  multiplication,  as  well  as  inversion. 
It  seems  appropriate  then  to  use  the  title,  "Errors  of  Matrix  Computations",  even  though  the  major 
:  effort  of  the  paper  is  devoted  to  the  question  of  the  errors  of  the  solutions  of  simultaneous  linear  equations, 
and  the  closel}^  related  problem  of  the  errors  of  the  calculated  inverse  matrix. 

Types  of  Errors 

Von  Neumann  and  Goldstine  [8:  1023-1032]  have  discussed  the  errors  involved  when  a  problem  in 

i  pure  or  applied  mathematics  is  solved  by  numerical  computation.     These  errors  are  errors  due  to  theory, 

errors  due  to  observation,  errors  of  approximation  or  truncation,  as  well  as  rounding-off  errors.     It  is 

not  my  purpose  to  discuss  all  of  these  errors.     I  direct  your  attention  to  the  situation  in  which  the  theo- 

j  retical  statement  of  the  problem  is  accepted,  but  the  mathematical  formulation  of  the  problem  involves 

'  coefficients  which  arc  subject  to  unspecified  but  limited  error.     For  example,  we  wish  to  discuss  the 

.  errors  of  the  solution  of  a  set  of  simultaneous  linear  equations  whose  coefficients  are  subject  to  error. 

We  do  not  question  the  suitability  of  the  linear  statement  of  the  problem,  but  we  are  concerned  (1)  with 

the  effect  of  the  original  errors  which  are  inherent  in  the  statement  of  the  problem  and  (2)  with  the 

:  cumulation  of  the  errors  which  result  from  the  continued  use  of  approximate  methods,  if  approximate 

methods  are  necessary.     We  wiU  call  these  latter  errors  the  computational  errors,  and  we  will  follow 

'Milne  [19:  30]  in  calling  the  original  errors  the  inherent  errors. 

'         As  an  illustration  we  consider  the  matj-ix  form  of  simultaneous  linear  equations  with  coefficients 
subject  to  error  as 

[a  +  e(a)]x=f+e(f),  (1) 


'  I'roffssor  of  Mathematics,  I'nivoisity  of  Michigan. 

-  Figures  in  brackets  indicate  the  literature  references  on  p.  58,  and  nunitiers  follow  inj;  tlie  colon  indicate  page  number. 
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where  x  and/  are  column  vectors,  and  where  e(a)  and  e{f)  &Te  the  inherent  errors  of  the  elements  of  the 
matrices  a  and/,  respectively.  If  e(a)  and  €(/)  are  known,  there  is  in  general  a  unique  solution  to  (1). 
However,  if  approximate  methods  are  used  in  the  solution,  computational  errors  will  be  introduced  so 
that  the  proposed  solution  for  x  will  not  satisfy  (1)  exactly.  The  cumulation  of  these  computational 
errors  has  been  a  chief  concern  of  several  recent  writers. 

In  the  more  typical  problem  the  values  of  e(a)  and  e(f)  are  not  knowTi  but  are  limited  by  some 
bounds  (which  are  usually  small  compared  with  the  elements  to  which  they  correspond).  In  this  case 
it  is  essential  that  we  learn  what  effect  these  unspecified  but  limited  errors  have  on  the  values  of  the 
resultant  solution. 

We  thus  have  two  problems  confronting  us:  (1)  What  can  we  say  about  the  effect  of  the  inherent 
errors  on  the  solution?  (2)  What  can  we  say  about  the  effect  of  the  cumulations  of  the  computational 
errors  on  the  solution  if  approximate  methods  are  used? 

In  many  situations  the  inherent  errors  as  well  as  the  computational  errors  result  from  rounding 
off  so  that  these  two  types  of  errors  are  treated  simultaneously  as  rounding-off  errors.  It  is  a  chief 
contention  of  this  paper  that  this  should  not  be  done.  The  two  types  of  errors  should  be  treated  separately 
insofar  as  that  is  possible,  and  the  results  may  then  be  integrated  to  provide  a  suitable  answer  to  the 
mathematical  problem. 

Use  of  Significant  Digits 

We  limit  our  discussion  to  finite  real  numbers.  We  further  limit  our  discussion  to  digital  numbers. 
It  is  not  true  that  every  real  number  can  be  written  in  pure  digital  form,  but  every  real  number  can  be 
put  in  the  form 

N=N-\-e,  (2) 

where  iV  is  a  digital  number.  Furthermore,  e  can  be  replaced  by  a  digital  positive  number  -q,  whose 
value  is  equal  to  or  greater  than  the  absolute  value  of  e.  Thus  every  finite  r_eal  number  can  be  approx- 
imated by  a  digital  number  with  its  digital  bound.  The  digital  number  A^  and  the  digital  bound  t; 
locate  the  real  number  on  the  interval  N—v  to  A^+77.  Hence  any  nondigital  real  number  can,  with 
some  rounding  off,  be  written  in  the  same  form  as  an  approximate  number,  that  is,  one  whose  value  is 
known  to  be  in  the  scale  interval  bounded  by  N—ri  and  N-hv- 

There  is  nothing  new  about  this,  of  course,  for  it  is  the  very  basis  of  the  conventional  use  of  significant 
figures.  When  using  significant  figures  the  convention  adopted  is  that  the  ??  is  one-half  of  the  last 
recorded  decimal  unit,  so  that  the  value  of  A^^  appears  but  not  the  bound.  This  leads  to  a  simple  com- 
puting technique  using  digital  numbers,  but  it  is  a  very  unsatisfactory  technique  because  the  convention 
limits  many  of  the  approximate  numbers  which  should  be  used  in  expressing  precise  though  approximate 
results  to  a  small  subclass  of  all  approximate  numbers.  No  approximate  number  which  has  a  bound 
greater  than  one-half  in  the  last  recorded  decimal  position  is  permitted. 

An  approximate  number  has  two  components,  and  any  trick  that  is  used  to  restrict  the  number  to 
one  component  by  holding  the  other  component  constant  is  very  unsatisfactory.  The  conventional 
rules  for  working  with  significant  digits  [13:27-33,  14:2-24]  are  so  unsatisfactory^  that  they  provide 
no  guarantee  that  the  results  of  a  continued  series  of  arithmetic  operations,  such  as  one  finds  in  the 
problem  of  the  solution  of  simultaneous  equations,  will  have  any  meaning  at  all. 

Further  study  with  more  adequate  s^'stems  of  approximate  numbers  shows  that  the  unsatisfactory 
results  of  computations  using  numbers  with  significant  digits  are  usually  due  more  to  the  deficiencies  of 
the  inadequate  tools  we  have  selected  to  work  with  than  they  are  to  the  inherent  indeterminacy  of  the 
problem. 

More  adequate  approximate  numbers  are  necessary  and  these  will  be  discussed  shortly.  First, 
there  are  one  or  two  things  which  I  think  we  might  learn  from  our  experience  with  computation  with 
significant  figures. 

Significant  figures  were  designed  to  handle  approximate  numbers  in  an  easy  way.  Perhaps  this 
was  in  order  before  we  had  machines  to  do  our  computing,  but  certainly  it  is  not  in  order  now.     There 
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IS  no  point  in  discarding  vital  information  just  to  simplify  the  computation,  but  that  is  what  we  are 
doing  continually  when  we  use  the  general  laws  of  computing  with  significant  digits. 

Computation  with  significant  digits  features  over-all  rules  by  which  the  number  of  significant 
.places  can  be  obtained  with  a  trivial  amount  of  computational  work.  Such  general  rules  are  not  of 
much  value  in  connection  with  a  specific  problem.  Considerable  additional  computational  work  is 
demanded  to  compute  the  two  components  of  each  approximate  number  needed  for  the  solution  of  the 
oroblem,  but  usually  the  results  are  very  much  more  precise  than  are  the  answers  obtained  by  the  rules 
pt  computation  with  significant  digits. 

We  have  given  up  the  formal  rules  of  computation  with  significant  digits  long  ago,  but  I  feel  we  are 
jtill  trying  to  get  over-all  rules  that  will  cover  the  accumulation  of  rounding-off  errors.  These  over-all 
rules  are  worth  while  in  understanding  the  nature  of  tha  processes  involved  but,  in  general,  they  should 
lot  be  expected  to  give  results  which  are  precise  enough  for  a  given  individual  problem.  If  we  have  a 
oroblem  with  coefficients  subject  to  error,  let  us  study  the  accumulation  of  the  errors  in  the  problem 
itself.  Of  course,  this  may  take  considerable  extra  computation,  but  that  is  the  way  to  get  the  precise 
mswer  to  our  problem,  if  we  want  it!  Besides,  we  arc  now  getting  simple  and  complex  machines  to  do 
:his  computing  for  us. 

I  propose  then  that  we  should  expect  that  specific  answers  to  a  problem  that  is  expressed  in  terms 
of  numbers  whose  coefficients  are  subject  to  error  should  be  answered  in  terms  of  the  specific  elements 
Df  the  problem.  The  practical  question  is  not  how  large  may  the  accumulation  of  the  errors  be  in 
oroblems  of  a  given  type  but  how  large  is  the  accumulation  of  errors  in  this  specific  problem! 

In  the  main,  the  formulas  in  the  later  sections  of  this  paper  feature  the  specific  problem  rather 
:.han  the  general  result.  If  we  have  to  compute  a  certain  inverse  matrix,  for  example,  in  order  to  answer 
;iertain  questions,  let's  compute  it! 

Types  of  Approximate  Numbers 

I  have  indicated  above  that  the  treatment  of  approximate  numbers  based  on  significant  figures 
s  very  unsatisfactory.  Any  satisfactory  representation  of  an  approximate  number  must  permit  the 
Precise  identification  (within  digital  limitation)  of  the  complete  approximate  number  as  an  interval 
m  a  scale.  This  can  be  done  by  giving  the  highest  and  lowest  points  of  the  interval,  these  being  the 
.wo  components  of  what  I  have  called  a  range  number  [13:12],  or  it  can  be  done  by  specifying  the  mid- 
Doint  of  the  interval,  which  we  might  call  the  point  approximation,  together  with  the  maximum  error 
Dossible  from  this  point  approximation.  This  number  I  have  called  an  approximation-error  number 
'13:12].  It  is  easy,  of  course,  to  change  from  range  numbers  to  approximation-error  numbers,  and 
Ace  versa. 

Modifications  in  these  basic  concepts  are  possible.  The  pure  range  numbers  may  be  replaced  by 
iducial  range  numbers  where  the  nature  of  the  problem  permits  the  estimation  by  probability  theory. 
This  concept  may  be  extended  to  include  statistical  approximate  numbers  where  the  approximate 
lumber  is  indicated  by  a  mean  and  a  variance  with  perhaps  some  distribution  law  for  the  unspecified 
^rror. 

The  approximation-error  numbers  may  be  replaced  by  first-order  approximation-error  numbers 
vlicn  first-order  error  terms  only  are  used  in  their  calculation. 

In  many  applied  problems  the  errors  are  not  known,  but  bounds  for  the  errors  are  known.  The 
errors  of  approximation-error  numbers  are  usually  given  in  terms  of  these  bounds.  It  is  proper  to  use 
•ounding  off  in  expressing  these  bounds  so  that  the  error  component,  as  well  as  the  point  approximation, 
•s  expressed  in  terms  of  digital  numbers.  These  dual  components  can  be  rounded  simultaneously  to 
1  fixed  number  of  places.  Thus  the  approximation-error  number  1.9684(186)  can  be  rounded  off  suc- 
i^.ively  to  get  1.968(19)  and  1.97(2). 

This  double  rounding-off  method  is  satisfactory  in  replacing  an  approximate  number  by  another 
ess  precise  approximate  number  which  does  not  differ  from  the  first  except  in  units  of  the  discarded 
ifigits.  This  is  also  satisfactory  if  first  order  approximation-error  numbers  are  used.  On  the  other 
'land,  if  one  wishes  to  guarantee  that  the  range  of  the  round ed-off  approximate  number  inchides  all 
>f  the  range  of  the  original  number,  he  may  express  the  original  number  in  range  form  and  then  round 
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off,  not  necessarily  to  the  nearest  digit,  but  so  as  to  incliule  the  original  range.     Thus  the  approximation- 
error  number  above 

1.9684(186)  ^[J;^^;^] 

,      ^  .     ,     ,    ri.987l       ,  ,,  ,    ri.99"l 

rounds  off  successively  to  and  thence  to     ^  ^^   • 

In  some  cases  the  point  approximation  of  the  approximation-error  number  alone  may  be  used. 
I  have  called  numbers  of  this  type  incomplete  approximation-error  numbers  [13:34],  or  incomplete  num- 
bers for  short,  to  point  out  the  fact  that  results  obtained  by  using  these  numbers  are  not  complete  until 
bounds  for  the  errors  are  added.  These  numbers  can  be  carried  to  a  fixed  number  of  places,  as  is  desirable 
in  machine  calculation,  without  implication  that  the  figures  recorded  are  significant  in  the  technical 
sense.  Some  computers  call  the  figures  of  these  numbers  "significant  figures"  and  reserve  the  term  "de- 
terminate significant  figures"  for  those  that  are  significant  in  the  technical  sense  [2].  The  incomplete 
numbers  are  exact  digital  numbers. 

Approximate   Matrices 

A  matrix  whose  elements  are  approximate  numbers  might  be  called  an  approximate  matrii.  The 
elements  of  these  matrices  might  appear  either  in  range  or  approximation-error  form.  In  general,  such 
an  m  by  n  approximate  matrix  is  composed  of  2mn  element  components,  since  each  of  the  elements 
in  an  exact  matrix  is  replaced  by  dual  components  in  the  approximate  matrix.  Approximate  matrices 
may  be  rounded  off  by  rounding  off  the  dual  components  of  each  element  according  to  the  methods 
described  in  the  last  section. 

A  matrix  whose  elements  are  incomplete  numbers,  that  is,  a  matrix  in  which  the  digital  point 
approximation  components  alone  appear,  might  be  called  an  incomplete  matrix.  The  complete  approxi- 
mate matrix  calls,  in  addition,  for  a  bound  for  the  error  component  for  each  term  of  the  matrix. 

Operations  With  Approximate  Matrices 

Rules  for  computation  with  approximate  numbers  in  range  and  approximation-error  form  are 
available  [13:  16-25],  although  recommended  rules  for  calculating  the  errors  of  approximation-error 
numbers  are  really  the  rules  of  differential  calculus  and  hence  lead  to  first-order  approximation-error 
numbers.  These  rules  can  be  applied,  of  course,  to  operations  with  matrices  whose  elements  are  ap- 
proximate numbers.  With  careful  attention  to  details  it  is  possible  to  establish  suitable  rules  for  sums, 
differences,  and  products  of  approximate  matrices  when  the  matrices  are  expressed  either  in  range  or 
approximation-error  form.  The  algebraic  operation  of  division  leads  to  the  matrix  operation  of  pre- 
or  post-multiplication  by  the  inverse.  The  problem  of  inverting  an  approximate  matrix  is  closely  re- 
late(i  to  that  of  the  solution  of  approximate  equations,  which  is  discussed  in  some  detail  in  the  later 
sections  of  this  paper. 

In  general,  operations  with  approximate  matrices  give  results  which  are  approximate  matrices 
with  guaranteed  bounds.  This  is  certainly  true  if  precise  methods  are  used  in  getting  the  bounds  at 
each  stage  of  the  computation.  It  is  approximately  true  when  first-order  approximation-error  methods 
are  used  in  the  computation  if  the  relative  error  of  each  calculated  term  is  small.  These  bounds  are 
not  guaranteed,  at  least  in  the  sense  used  above,  when  some  assumption  is  made  about  the  distribution 
of  the  true  error  in  the  approximation-error  numbers  and  statistical  formulas  are  used  to  estimate  the 
range  at  some  probabilit}'  level. 

The  strict  results  are  reasonably  satisfactory  when  the  problem  demands  continued  additions 
and  subtractions,  although  the  probability  approach,  under  suitable  assumptions,  may  lead  to  closer 
practical  bounds  [15],  even  though  these  bounds  can  not  be  shown  to  be  true  in  the  strict  sense. 

The  strict  results  are  not  so  satisfactory  for  operations  involving  continued  multiplications  though, 
theoretically,  all  multiplications  can  be  carried  out  with  the  use  of  range  numbers  or  approximation- 
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error  numbers,  aud  the  commutative  property  of  product  operations  is  preserved.  However,  an  exten- 
sive series  of  multiplications  leads  to  numbers  with  so  many  places  that  it  is  necessary,  from  a  practical 
standpoint,  to  round  off.  It  is  desirable,  in  such  cases,  to  round  off  to  a  fixed  digital  position.  The 
round-off  procedure  introduces  rounding-off  errors  that  differ  according  to  the  particular  product  in- 
volved, so  the  final  product,  when  the  intermediate  products  have  been  rounded  to  some  given  decimal 
position,  varies  with  the  order  in  which  the  different  factors  are  included  in  the  accumulating  product. 
Multiplication  by  approximate  numbers  in  appropriate  form  is  commutative,  but  it  is  not  commutative 
if,  for  the  sake  of  simplification,  we  indidge  in  rounding  off  in  the  process  of  computation.  In  a  loose 
sense,  we  might  say  that  commutativity  is  approached  as  the  rounding-off  errors  are  made  smaller  and 
■smaller. 

j  First-order  multiplications  with  approximation-error  numbers  are  not  in  general  commutative  since 
( second-order  terms  are  neglected  at  each  multiplication.  However,  if  the  relative  errors  are  small,  the 
I  second-order  relative  errors  are  very  small  and  the  difTerences  between  the  various  products  are  trivial. 
I  Formal  treatment  of  material  very  similar  to  this  is  given  by  von  Neumann  and  Goldstine  [8  :  10:55- 
'  1039],  in  their  discussion  which  shows  that  in  general  pseudo-operations  are  not  commutative. 

The  situation  is  somewhat  worse  with  respect  to  division  because  the  quotient  of  two  exact  digital 
numbers  is  not  necessarily  digital.    It  follows  that  the  point  approximation  of  the  ratio  of  two  approxi- 
mate numbers  is  not  necessarily  digital  but  must  itself  be  rounded  off  to  attam  digital  form.    It  is  obvious 
that  the  fuial  result  of  a  series  of  divisions  depends  upon  the  order  in  which  the  divisions  are  performed 
'  if  the  divisions  are  carried  to  a  fixed  decimal  position. 

Difficulties  of  a  much  larger  order  are  encountered  when  division  by  an  approximate  number  is 
replaced  by  the  inversion  of  an  approximate  matrix.    I  would  like  to  devote  the  most  of  the  remainder 
i  of  my  space  to  this  problem  and  to  the  allied  problem  of  the  numerical  solution  of  equations  with  coeffi- 
cients subject  to  unspecified  but  limited  error. 

Before  taking  up  this  topic,  I  would  like  to  make  one  remark  about  the  cumulation  of  inherent  error 

in  calculation.     Calculations  with  range  numbers  or  with  approximation-error  numbers  are  very  cum- 

;  bersome  when  numerous  successive  calculations  are  demanded.    It  is  preferable  to  calculate  incomplete 

numbers  and,  after  the  answer  is  given  m  terms  of  incomplete  numbers,  to  calculate  bouads  for  the 

answer  with  the  use  of  an  appropriate  error  formula,  if  such  a  formula  is  available.    Such  a  formula  can 

•  sometimes  be  derived  from  the  theoretical  statement  of  the  problem,  if  care  has  beea  taken  in  stating 

•  the  problem  so  that  the  inherent  errors  are  separated  from  the  calculational  errors.  This  general  method 
•is  illustrated  below  by  applying  it,  in  considerable  detail,  to  the  linear  equations  problem. 

I 

The  Solution  of  Approximate  Simultaneous  Equations 

Just  what  do  we  mean  by  the  solution  of  linear  simultaneous  equations  with  coefficients  subject  to 

!  unspecified  but  limited  error?    First,  what  do  the  equations  themselves  mean?    These  equations  do  not 

represent  just  one  set  of  equations,  but  a  multiple  infinity  of  sets  of  equations,  since  each  coefficient  that 

,  is  indicated  by  an  approximate  number  in  digital  form  actually  might  be  any  one  of -an  infinity  of  values. 

'  Thus  the  algebraic  equations 

(oi  I  +  «ii)a;i  +  (ai2  +  ei2)a;2  =  a^  +  €13 

(Cf'2I  +  €2l)a^l  +  (^22  +  C22)a;2  =  ^23  +  ^23 

represent  a  sextuple  infinity  of  sets  of  two  equations  hi  two  unknowns,  shice  each  etj  may  be  taken  on 
any  of  the  infinity  of  values  between  —  r?,^  and  r/,;. 

As  the  multiple  mfmity  of  the  sets  of  equations  can  only  be  represented  symbolically,  it  is  obviously 
impossible  to  write  all  the  multiple  infinity  of  solutions  explicitly.  What  we  can  do  is  provide  some  single 
solution  that  may  be  said  to  represent  the  multiple  infinity  of  all  solutions.  This  may  be  done  in  various 
ways. 

We  may  obtain  the  unique  solution  of  any  one  of  the  multiple  uifinity  of  sets  of  exact  equations  and 
use  that  solution  as  representative  of  all  the  solutions.    This  may  be  satisfactory  in  some  cases,  but  it 
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must  be  noted  that  the  solution  is  not  unique  and  that  the  answer  provides  no  measure  of  the  extent  to 
which  this  solution  may  differ  from  the  solution  of  some  other  set.  The  particular  set  of  equations  chosen 
may  have  a  solution  that  is  extreme,  so  that  the  solution  is  not  very  satisfactory  as  a  representative  one. 

An  arbitrary  unique  solution  that  may  have  some  special  merit  is  that  obtained  by  solving  the  set 
of  equations  in  which  all  the  error  terms  are  zero.  The  resulting  equations  are  unique  exact  equations, 
and  they  may  be  solved  to  any  desired  degree  of  accuracy.  The  results  are  incomplete  numbers,  but 
these  may  be  said,  in  a  unique  sense,  to  represent  all  the  multiple  infinity  of  solutions.  This  type  of 
solution,  unless  modified,  is  also  quite  unsatisfactory  because  there  is  no  indication  of  the  extent  to  which 
the  other  solutions  may  differ  from  this  one.  We  might  call  this  the  incomplete  solution,  since  it  is  the 
unique  solution  of  the  incomplete  equations. 

A  more  satisfactory  type  of  solution  is  that  in  which  the  answers  are  given  as  approximate  numbers. 
Not  only  is  the  point  approximation  given,  but  some  estimate  of  the  variation  of  the  answer  is  provided. 

With  the  use  of  suitable  formulas  derived  separately,  and  with  additional  computation,  we  can  pro- 
vide bounds  for  the  answer  to  the  incomplete  solution  just  described.  This  can  be  done  approximately 
with  the  use  of  first-order  error  terms  or,  more  precisely,  with  higher  order  error  terms.  The  extrema  of 
the  various  Xi  can  be  computed  or,  if  desired,  the  values  of  the  other  Xj  associated  in  a  solution  with  each 
extreme  Xt  can  be  computed. 

i'-  Under  suitable  assumptions,  probability  formulas  can  be  used  to  indicate  the  expected  deviation  of 
a  typical  solution  from  the  incomplete  solution. 

In  consideration  of  the  above  discussion  it  seems  desirable  to  express  the  solution,  as  well  as  the 
coefficients,  in  terms  of  approximate  numbers.  The  mathematical  formulation  of  (1)  might  then  be 
modified  to  allow  for  the  error  in  the  solution  and  would  appear  as 

[a  +  e(a)][xi-e(x)]=f+e(f),  (3) 

where  x  indicates  the  incomplete  solution,  which  is  the  solution  of  the  mcomplete  equation 

ax=f,  (4) 

if  a  is  nonsingular,  and  can  be  indicated  by 

x  =  a-'J.  (5) 

Since  the  elements  of  a  and /are  exact  digital  numbers,  it  fofiows  that  the  value  of  each  Xt  can  be  written 
as  the  ratio  of  two  digital  numbers,  as  is  evident  from  an  application  of  Cramer's  Rule. 

Simultaneous  solution  of  the  matrix  eq  (3)  and  (4)  enables  us  to  get  a  theoretical  expression  for 
e(x).     We  get 

ae{x)-{-€{a)x^e{a)e(x)=€(f)  (6) 

so  that 

[a  +  e(a)]e(x)  =  €(J)-e(a)x  (7) 

e(x:)  =  [a-he(a)]''[e{f)~e{a)x].  (8) 

A  series  expansion  of  (8)  results  m  the  formula: 

e(x)={l-a-hia)^[a-'e(a)Y~[a-'e(a)f+.  .  .}a-'[e(f)-e(a)x],'  (9) 

which  is  similar  to  matrix  formulas  used  by  Lonseth  [3],  Turing  [10],  and  Waugh  [16]. 

The  formula  provides  the  theoretical  answer  for  the  errors  of  the  solution  if  the  original  inherent 
errors  are  known  and  are  relatively  small.  It  can  be  used  as  the  basis  of  a  formula  for  the  bound  of 
the  errors  of  the  solutions  when,  as  is  usual,  the  inherent  errors  are  indicated  by  then-  bounds.  Various 
formulas  for  bounds  are  discussed  later.  First,  we  examme  the  question  of  the  solution  of  the  incom- 
plete equations. 

5  Valid  under  favorable  conditions  of  convergence. 
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The  Solution  of  the  Incomplete  Equations 

The  theoretical  solution  of  the  incomplete  equation  (4)  is  very  simple,  if  a  is  nonsingular,  but  the 
numerical  solution  becomes  complicated  if  many  variables  are  involved.     The  numerical  solution  of  these 
equations  seems  to  have  received  more  attention  during  recent  years  than  has  any  other  phase  of  the 
general  problem.     A  chief  interest  has  been  the  accumulation  of  the  rounding-off  errors  with  approximate 
'  methods  [5]  and  the  fact  that  in  general  no  guarantee  can  be  given  that  these  cumulations  are  not  enor- 
mous [5].     I  should  also  mention  studies  of  particular  methods  with  a  view  to  selection  of  a  method  in 
which  the  cumulation  of  rounding-off  errors  will  be  a  minimum  [9,  10],  the  use  of  iteration  as  a  means 
of  improving  the  accuracy  which  may  be  destroyed  by  successive  round  offs  [5,6],  the  use  of  error  control 
I  in  which  matrix  manipulations  are  applied  to  the  original  matrix  equation  so  as  to  reduce  it  to  a  form  in 
t  which  the  round-off  accumulations  are  laiown  to  be  relatively  trivial  [7],  the  use  of  functions  of  the 
(  elements  of  a  matrix  such  as  the  norm  in  obtaining  bounds  for  the  accumulation  of  the  rounding-off 
;  error  [5,  6,  7,  8,  10,  11],  the  use  of  scale  adjustments  to  bring  the  elements  of  the  matrix  into  comparable 
flanges  [8:  1034;  17:  30;  18],  etc. 

!  I  would  like  to  discuss  these  in  detail,  but  space  permits  only  a  passing  reference  to  these  important 

results  and  techniques. 

;  Aly  experience  has  led  me  to  the  conclusion,  and  there  are  some  others  who  have  arrived  at  the  same 

'  conclusion  [10],  that  direct  elimination  methods  are  for  many  problems  best.     Certain  types  of  problems 
I  converge  quickly  with  iterative  methods,  and  we  do  not  know  yet  just  wdiat  methods  we  will  use  when 
I  we  solve  equations  in  very  many  variables  with  various  electronic  digital  computers  but,  with  present 
problems  and  equipment,  direct-pivoted  methods  seem  not  too  unsatisfactory  if  we  can  handle  the 
problem  of  the  accumulation  of  errors. 

1  WTiere  the  number  of  variables  is  relatively  small,  I  like  to  use  exact  pivotal  elimination  methods 

!  [13:  50-89].     There  is  some  theoretical  advantage  in  using  a  modified  form  of  an  exact  method  when 
'  the  number  of  variables  is  fairly  large  [13:  88],  since  each  calculated  value  can  be  interpreted  as  a  de- 
terminant. 
;  It  is  perhaps  preferable  to  select  the  pivots  from  the  diagonal  terms  since  they  are  the  nonzero 

multipliers  of  principal  minors,  and  it  is  helpful  to  know  that  these  principal  minors  do  not  vanish. 
For  present  purposes,  I  assume  that  some  approximate  solution  to  (4)  can  be  obtamed  with  the  use 
'  of  some  one  of  the  various  methods  described  by  Forsythe  [12].     It  is  desired  to  loiow  how  satisfactory 
1  this  approximate  solution  is.     It  is  not  necessary  that  the  solution  be  exact,  but  it  is  important  that 
the  errors  of  the  solution  of  the  incomplete  equation  are  not  larger  than  the  accumulation  of  the  inherent 
'  errors. 

We  can  substitute  the  proposed  solution  in  the  left  hand  side  of  the  incomplete  equations  to  see 
how  well  the  equations  are  satisfied.     If  we  let  e  indicate  the  matrix  of  computational  errors,  we  can 
'  write 

a[x-\-e(x)]=f+e(f).  (10) 

If  the  values  of  e(f)  are  within  the  bounds  specified  for  e(/)  we  may  say  in  general  that  the  solution  is 
satisfactory,  unless  the  equations  are  ill-conditioned.  Methods  of  studying  the  extent  of  ill-conditioning 
of  a  matrix  have  been  developed,  see,  for  example  [10],  but  it  is  my  thesis  that  ill-conditioning  will  in 
general  be  revealed  if  direct  elimination  methods  are  used  with  diagonal  pivots  and  certainly  if  the 
inverse  matrix  is  calculated.  In  fact,  formulas  for  ill-conditioning  used  by  Turmg  [10]  feature  the 
elements  of  the  inverse. 

If  the  values  of  e  are  too  large,  it  is  possible  to  follow  the  iterative  methods  of  IIo telling  [6],  the 
error  control  method  of  Satterthwaite  [7],  or,  to  use  the  approximate  inverse,  Cq,  of  a  as  the  basis  of  a 
further  approximation.     Substitution  of  the  fact  that  ax=f  m  (10)  yields  a€(x)=e(f),  so  that 

e(x)=a-'e(f)  (11) 

with  o~^  =  Co+e(co).     We  have  e(x)=Coe{f)+e(co)e(f)  and 

e(x)=Coe(f)  (12) 
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as  the  first-order  approximation.     Other  methods  could  be  used,  but  this  uses  the  approximate  inverse. 
It  should  be  noted  that  all  of  these  methods,  including  the  examination  of  ill-conditioning,  call  for 
the  calculation  of  the  inverse  matrix  or  an  approximation  to  it. 

Bounds  for  the  Inherent  Errors  of  the  Solutions 

Bounds  can  be  found  for  the  errors  indicated  by  (9).  If  we  omit  the  error  terms  of  order  higher 
than  one,  we  have 

t{x)=a-'[€{j)-t{a)x].  (13) 

We  get  values  for  the  bounds  of  t{Xk)  when  the  absolute  values  of  all  the  terms  in  (13)  are  taken 
and  the  t{a)  and  tij)  are  replaced  by  their  bounds.  For  example,  if  -q  is  the  bound  for  all  the  elements 
of  e(a)  and  «(/),  we  obtain  the  formula 

77(a-.)=77(i+i:i2:,i)z;ia,ri>  ^i^) 

where  the  pairs  of  vertical  lines  indicate  absolute  values.  This  formula,  or  formulas  very  similar  to  it, 
have  been  given  by  several  authors  [1,  3,  13:262]. 

It  should  be  pointed  out  that  the  first-order  answer  to  the  problem  is  now  stated  in  terms  of  the 
values  of  Xi,  which  are  the  solutions  of  the  (exact)  incomplete  equations,  and  the  elements  of  a~\  which 
is  the  inverse  of  an  exact  matrix.  This  is  very  desirable  because  the  formula  gives  bounds  for  the  effect 
of  the  inherent  errors  which  do  not  involve  the  computational  errors.  To  be  sure,  we  use  values  which 
are  approximations  to  those  of  Xi  and  a~^,  but,  in  practice,  these  approximations  need  be  carried  only 
far  enough  to  fix  the  first  decimal  positions  of  ■t){xk). 

The  formula  (14)  does  give  first-order  bounds  for  the  extreme  errors  of  the  different  Xk,  but,  in  a 
real  sense,  it  does  not  give  solutions  since  ri{xk)  is  not  necessarily  associated  with  -niXj).  It  is  possible  to 
return  to  (13)  and  to  assign  values  to  the  e{f)  and  the  e(a)  in  such  a  way  as  to  maximize  some  particular 
rj{xic),  and,  by  substitution  m  (1)  to  produce  the  actual  equations,  aside  from  errors  of  order  higher 
than  the  first,  which  have  these  extreme  solutions.  Techniques  that  use  the  inverse  or  adjugate  matrix 
are  available  for  determining  these  sets  of  extreme  equations  [13:  264-276]. 

The  formulas  above  use  the  inverse  matrix.  Some  authors  prefer  to  find  bounds  for  first-order 
errors  without  the  calculation  of  the  inverse  matrix.  Among  these  are  Milne  [19:29-35]  and  Willers 
[20:271-273],  who  use,  in  effect,  the  formula 

ae{x)  =  t{j)-e{a)x,  (15) 

which  is  (13)  premultiplied  by  a.  The  right-hand  side  is  replaced  by  its  bound  (l  +  Slxj)??.  Willers 
and  Milne  use  different  methods  of  solution  in  guaranteeing  the  bounds  and  in  general  arrive  at  bounds 
that  are  much  larger  than  they  need  be.  These  bounds  become  very  much  larger  than  the  least  upper 
bound  if  a  back  solution  is  used.  It  is  wise,  if  close  bounds  are  desired,  to  carry  through  a  new  elimination 
process  for  each  Xk,  and,  if  this  is  done,  one  might  as  well  calculate  the  inverse  matrix  and  use  the  adequate 
bounds  that  this  method  provides. 

The  formula  (14)  simplifies  somewhat  when  all  the  e(/),  all  the  e(a),  or  certain  of  the  values  of  e(a), 
are  zero  [13:278-284]. 

Higher  order  approximations  to  the  errors  can  be  obtained  Wii\\  the  use  of  more  of  the  terms  of  (9). 
Thus  the  second  approximation  is 

e{x)  =  {I-ar't{a)]a-Mi)-t{a)x] 

=a-'U{j)-e{a)x\-a-h{a)a-'U{J)-t{a)x].  (16) 

Bounds  can  then  be  taken  as  before. 

Bounds  for  second-order  errors  have  been  provided  by  Lonseth  [3],  who  has  also  used  a  recursion 
formula  and  mathematical  induction  in  estimating  bounds  for  higher  order  errors.  Lonseth  [3]  and 
Etherington  [1]  have  made  some  study  of  suitable  probability  distributions  of  these  errors. 
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Tlie  Errors  of  the  Inverse 

The  discussion  of  the  errors  of  the  inverse  is  rather  brief,  because  the  calcuhition  of  the  inverse  is, 
,  essentially,  a  special  case  of  the  more  general  problem  of  the  solution  of  sets  of  simultaneous  equations. 
I  It  results  from  solving  the  equation 

j  [a  +  e(a)][x  +  eix)]  =  I.  (17) 

[The  identity  matrix  is  not  subject  to  error  so  the  problem  is  somewhat  less  complicated  than  the  previous 
I  one.     We  obtain  the  incomplete  solution  from  the  exact  incomplete  equations 

j  ax=I  (18) 

as  before  and  have 

€(a)x  +  a€(x)  +  e(a)e(x)  =  0.  (19) 

It  follows  that 

I  e{x)=-[a-^e{a)]-'eia)x=-[a  +  e(a)]-'e(a)a-\  (20) 

This  is  expanded  to  get 

e{x)  =  -[I+a-'e{a)]-'a-'t{a)a-K  (21) 

The  first-order  approximation  is 

e(a-')=-a-'€{a)a-K  (22) 

The  formula,  which  is  a  matrix  generalization  of  the  formula  for  the  differential  of  x~\  expresses  the 
i  inherent  errors  of  the  inverse  in  terms  of  the  original  inherent  errors  and  the  elements  of  the  incomplete 

inverse,  which  appears  twice  in  the  numerator. 

Bounds  can  then  be  determined  for  each  of  the  elements  of  the  inverse  from  formulas  that  feature 
'the  absolute  values  of  terms  of  the  incomplete  inverse  [13:284-288]. 

Suiiiiiiary  and  Conclusions 

This  paper  results  from  an  attempt  to  study  the  nature  of  the  general  mathematical  problem  of 

[the  solution  of  approximate  simultaneous  equations  and  to  indicate  theoretical  and  practical  solutions 

[by  analyzing  the  problem  in  such  a  way  that  the  effects  of  the  inherent  errors  are  separated  from  the 

computational  errors.     The  presentation  features  references  to  much  important  work  that  has  been 

done  on  several  phases  of  the  problem.     Much  of  this  basic  work,  which  is  barely  mentioned,  deserves 

more  detailed  consideration  than  the  time  limits  of  this  paper  have  permitted. 

A  general  discussion  of  the  nature  of  the  errors  involved  is  followed  by  an  examination  of  suitable 
means  of  expressing  the  results.  A  matrix  presentation  of  the  problem  is  useful,  if  the  elements  of  the 
matrix  are  range  numbers  or  approximation-error  numbers.  The  single-component  numbers  that 
jfeature  significant  digits  are  not  satisfactory. 

i  Extensive  computations  with  range  numbers,  or  with  approximation-error  numbers,  are  unsatis- 
i factory,  in  practice,  when  long  series  of  operations  are  demanded,  as  in  the  solution  of  linear  equations 
with  pivotal  methods.  The  necessity  of  rounding  off,  and  the  alternative  selection  of  pivots,  leads  one 
to  ranges  and  bounds  that  are  not  as  close  as  is  possible,  although  they  include  the  answers. 

These  difficulties  are  largely  avoided  with  the  use  of  incomplete  numbers  when  a  separate  formula 
for  determining  the  error  bounds  is  available.  The  matrix  statement  of  the  problem  of  linear  equations 
lis  broken  down  into  an  exact  matrix  equation  whose  iniknowns  are  Xt,  the  values  of  the  incomplete 
; solution,  and  another  matrix  equation  whose  unknowns  are  the  i(xi),  the  values  of  the  inherent  errors 
'of  the  solution.  The  formula  for  e(Xi)  features  the  inverse  of  the  matrix  of  the  coefficients,  a-\  of  the 
incomplete  equations,  as  well  as  the  inherent  errors,  tij,  of  the  coefficients. 

Any  approximate  solution  of  the  incomplete  equations  will  result  in  values  x<+e(a;<),  where  Xi  is  the 
.exact  value,  and  e(Xi)  is  the  accumulation  of  the  rounding-off  errors.  From  the  practical  standpoint, 
ja  reasonably  satisfactory  solution  is  obtained  when  e{Xi)  is  appreciably  less  than  T]{Xi),  the  value  of  the 
bound  for  €{Xi).     Less  precisely,  an  approximation  may  be  considered  to  be  satisfactory  when  e(J)  is 


very  small.  A  more  satisfactory  first-order  approximation  for  e{x)  may  be  obtained  by  premultiplying 
e{j)  hj  Co  an  approximation  to  a~^ 

The  formulas  of  this  paper  are  specific  formulas  in  that  they  feature  the  elements  of  a-\  They 
demand  the  calculation  of  a'^  or  of  some  good  approximation  to  it.  Rough  approximations  are  adequate 
for  certain  practical  problems,  and  over-all  rules  are  then  in  order,  but  we  should  clearly  distinguish 
such  over-all  rules  from  mathematically  precise  solutions. 

I  hope  that  these  remarks  will  help  to  clarify  some  of  the  various  questions  that  arise  in  the  study 
of  simultaneous  linear  equations  with  unspecified  but  limited  error. 
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7.  Rapidly  Converging  Iterative  Methods 
for  Solving  Linear  Equations 

J.  Barkley  Rosser  * 

1.  In  the  fall  of  1949  the  Institute  of  Numerical  Analysis  of  the  National  Bm-eau  of  Standards 
mdertook  an  extended  study  of  methods  of  solving  simultaneous  linear  equations.  Tkrough  the  use 
Df  colloquia  and  discussion  groups,  nearly  all  scientific  members  of  the  Institute  haye  made  some  sort  of 
contribution  to  the  problem.  Accordingh',  it  is  impossible  to  assign  complete  credit  for  the  results  dis- 
■?losed  herein  to  a  single  person  or  a  fe^y  persons.  However,  certain  members  of  the  staff  have  given  con- 
centrated attention  to  the  problem  over  an  extended  period  and  are  primarilj-  responsible  for  the  results 
acted  herein.  In  alphabetical  order,  these  are  G.  E.  Forsythe,  M.  R.  Hestenes,  C.  Lanczos,  T.  Motzkin, 
L.  J.  Paige,  and  J.  B.  Rosser.     Of  these,  the  last  is  serving  as  expositor  in  the  present  paper. 

2.  Finding  the  center  of  an  ellipsoid.  Suppose  we  have  an  ellipsoid  in  n  dimensions.  Consider  the 
liamily  of  ellipsoids,  concentric  with,  similar  to,  and  similarly  oriented  with  the  given  ellipsoid.  Any 
3ne  of  these  can  be  considered  as  generated  from  the  given  ellipsoid  by  expansion  (or  contraction)  from 
the  center  in  a  given  ratio. 

I  We  first  outline  a  theoretical  scheme  for  finding  the  common  center  of  these  ellipsoids  in  n  steps. 
(We  give  the  computational  details  later.)  We  note  that  for  am'  direction  2,  there  is  a  conjugate  hj-per- 
plane  Pi  defined  as  the  hj'perplane  conjugate  to  the  point  at  infinity  along  the  direction  2,.  Given  2,, 
jthe  plane  Pi  is  the  same  for  each  ellipsoid  of  our  family,  and  for  each  ellipsoid  of  the  family  the  hyperplane 
Pi  bisects  all  chords  having  the  direction  Zi.  In  particular,  Pi  passes  through  the  center  of  aU  the  ellip- 
soids. 

Suppose  we  determine  a  series  of  directions  Zi,  .  .  .,  2„,  and  their  conjugate  hyperplanes  Pi, 
„,  with  the  property  that  Zi  is  parallel  to  each  of  Pi,  .  .  .,  Pf_i.  Then  we  can  determine  the  center 
of  the  ellipsoids  b}'  the  foUoA^ing  construction.  Choose  a^o  any  point.  From  Xo  proceed  in  the  direction 
jsi  until  the  hyperplane  Pi  is  reached.  Designate  this  point  Xi.  Now,  from  Xi  proceed  in  the  direction 
■22  until  the  h3'perplane  P2  is  reached.  Designate  this  point  Xz.  Proceed  in  this  manner,  getting  a:,+, 
from  Xi  by  proceeding  in  the  direction  Zi+i  until  the  hyperplane  Pi+i  is  reached.  One  easily  proves,  b}'^ 
imduction  on  i,  that  Xi  is  on  each  of  Pi,  .  .  .,  P,.  However,  the  only  point  common  to  each  of  Pi, 
P„  is  the  center  of  the  ellipsoids.     So  a;„  is  this  center. 

3.  Algebraic  method  of  finding  the  center.  Let  c  be  a  point  and  C  a  real  symmetric  matrix.  Then, 
I'or  positive  real  values  of  <!>,  the  surfaces 

(x-cyC{x-c)  =  <l>  (1) 

Iform  a  family  of  ellipsoids  such  as  considered  above,  with  tlie  common  center,  c.  Indeed,  each  such 
ifamih'  of  ellipsoids  can  be  so  represented. 

If  then  Zi  is  a  direction,  the  conjugate  hyperplane  Pi  consists  of  all  points  lu,  such  that 

{w—cyCZi=0.  (2) 

'En  particular,  a  direction  Zj  is  parallel  to  Pi  if,  and  only  if, 

zfCZi=0         (i9^j).  (3) 


We  may  express  (3)  in  words  by  saying  that  Zj  and  Zi  are  C-orthogonal.  Then  if  we  wish  2<  to  be 
[Darallel  to  each  of  Pi,  .  .  .,  P,-i,  we  need  only  to  insure  that  Zi  is  C-orthogonal  to  each  of  2,,  .  .  ., 
J<_i.  Since  C-orthogonality  is  a  symmetric  relation,  we  see  that  tiie  condition  we  wish  the  2i,  .  .  .,  z„ 
:o  satisfy  is  that  they  be  C-orthogonal  in  pairs. 


•  National  Bureau  of  Standards,  Los  Angeles,  California,  and  Cornell  University. 
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One  can  find  such  a  set  of  z's  by  various  means.  One  scheme  is  essentially  a  Gram-Schmidt  pro- 
cedure. Choose  Wi,  .  .  .,  Wn  &  set  of  independent  directions.  Now  choose  Zi  =  Wi.  Then  take  22  to 
be  of  the  form  W2  —  aZi.  One  readily  chooses  a  so  that  22  is  C-orthogonal  to  z^.  Then  take  23  to  be  of  the 
form  W3—aZi  —  ^Z2.     One  readily  chooses  a  and  /3  so  that  23  is  C-orthogonal  to  each  of  z^  and  22.    And  so  on. 

A  somewhat  simpler  scheme  is  as  follows.^  Take  Zi  to  be  any  arbitrary  direction.  Take  22  to  be 
of  the  form  Czi—aZi,  choosing  a  so  that  22  is  C-orthogonal  to  z^.     Thereafter,  take  Zt+i  to  be  of  the  form 

Zi+i  =  CZi  —  aiZi  —  ^iZi-i,  (4) 

choosing  at  and  /S,  so  that  2^+1  is  C-orthogonal  to  each  of  Zi  and  2,_i.     Then  2^+1  is  C-orthogonal  to  each 
of  2i,  .  .  .,  Zi.     This  is  a  special  case  of  the  following  theorem. 

Theorem  1.     Let  B  and  C  he  symmetric  matrices  which  commute.     Let  2o=0,  Zx  he  arhitranj  and 


'i+V 


CZi  —  aiZi  —  ^iZi^i 


for  i>  1 ,  with  ai  and  I3i  chosen  so  that  Zf^i  is  B-orthogonal  to  each  oj  Zi  and  Zt^i.      Then  2i+i  is  B-orthogonal  . 
to  each  of  Zq,  Zi,  .  .  .,  Zf. 

Proof  hy  induction  on  i.  Since  2o=0,  it  is  5-orthogonal  to  all  2,.  So  our  conclusion  holds  for  i  =  Q. 
Let  us  assume  the  conclusion  for  i.  By  our  choice  of  at  and  /S^,  2442  is  5-orthogonal  to  each  of  2i_,_i  and 
Zi.  It  is  also  5-orthogonal  to  Zq.  Consider  zJBzi^2,  with  0<Ci<Ci.  By  the  hypothesis  of  our  induction, 
Zj  is  5-orthogonal  to  each  of  2,^.1  and  2j.  So  by  (4)  and  the  assumption  that  B  and  C  are  S3^mmetric 
matrices  which  commute, 

zfBzi+2=2!BCZi+,  =  zfCBzi+,  =  z!^,BC%: 
By  (4)  again, 

zJ'BZi+2  =  zJ+,Bzj^i^ajz'[+^Bzj^^jzf^^Bzj_,  =  0. 

Thus  Zj  is  i?-orthogonal  to  2i+2,  and  our  theorem  is  proved. 

In  theory,  this  scheme  can  terminate  by  having  Zi=0  for  i<n.  In  a  typical  computation,  there  is 
a  zero  probability  of  picking  a  Zi  that  causes  the  series  to  terminate  too  soon,  and  we  may  ignore  the 
possibility.  In  the  application  to  solving  simultaneous  equations  given  below,  one  would  have  the 
solution  whenever  one  gets  to  2^=0,  and  so  the  choice  of  an  initial  2i  is  truly  immaterial. 

Once  one  has  the  2's,  one  easily  gets  the  x's.    For,  given  Xi,  one  can  choose  <^  in  (1)  so  as  to  get  the  i 
ellipsoid  of  our  family  which  passes  through  Xi.    Now  Xi+i  is  halfway  across  this  ellipsoid  in  the  direction 

4.  Solution  of  simultaneous  equations.  Let  ^  be  a  nonsingular  matrix  and  k  a  vector,  and  suppose 
we  wish  to  solve 

-      Ax=k.  (5) 

Take  H  any  real  symmetric  matrix.     Then  for  real  positive  (j) 

(Ax  -  k)  ""HiAx  -k)=4>  (6) 

is  one  of  a  family  of  similar,  similarly  oriented  ellipsoids  with  center  .4"'^-.    That  is,  we  can  write  (6)  in 
the  form 

{x-A-'kYA''HA{x-A-'k)  =  4>. 

Then,  comparing  with  (1),  we  have  c  =  A~^k,         C=A'^HA. 

Then  the  methods  outlined  in  the  preceding  section  suffice  to  determine  the  center  of  (6),  which  is 
identical  with  the  solution  of  (5) . 

In  the  present  case,  we  can  determine  Xn  directly  by  the  following  device.  By  our  method  of  suc- 
cessive generation  of  the  cc's,  we  see  that  a^i  has  the  form  aro+ai2i+.  .  .+aiZi.  Recalling  that  2;„  is  .4"^A:, 
we  see  that  we  wish  values  of  a,  such  that 

»  C.  Lanezos,  An  iteration  method  for  the  solution  of  the  eigenvalue  problem  of  linear  differential  and  integral  operators,  J.  Research  NBS  45,  255-282  (1950); 
RP  2133  (see  section  VII). 
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Multiplying  on  the  left  by  zf A'^ HA,  &nd  recalling  that  the  z's  are  C-orthogonal  (with  C=A'^HA), 
jeget 

zfA^Hk=zjA''HAxo+ajzjA^HAzj,  (7) 

hich  determines  aj. 

One  should  note  that  for  the  above  procedui-e,  one  requu-es  only  a  laiowledge  of  A^H,  A'^HA,  k,  Zi, 
(id  Xq,  of  which  H,  Zi,  and  Xq  are  at  our  disposal.  In  case  A  is  symmetric  and  positive  definite,  one  can 
j,ke  H=A~^,  with  considerable  simplification  of  the  computations.  Otherwise,  H^I  is  a  convenient 
iioice. 

5.  First  iterative  scheme.  If  one  has  any  set  of  z's  that  are  C-orthogonal,  it  clearly  suffices  to  replace 
lem  by  any  multiples  of  them.     Thus,  w^e  can  generalize  (4)  to 

Zi+i  =  yiiCZi—aiZi—^iZi^i).  (8) 

ji  case  A  is  symmetric  and  positive  definite,  we  take  H=A-\  so  that  C=A.    Then  the  following  scheme 
i'oceeds  according  to  (8) . 
We  define  the  residuals 

ri=k-Axi,         {0<i).  (9) 

We  note  that  our  scheme  for  generating  the  x's  is  expressed  by 

Xi=Xi_i+ai2i,         (l<i),  (10) 


(l<i).  (11) 


here  < 

T 

z,  ri_, 

2.-42,- 

'jy  (9)  and  (10), 

r,=r,_,-a,Az„         (l<i).  (12) 

By  use  of  this,  one  can  simplify  (8)  to 

2i+,  =  r,+6,2„         (0<i),  (13) 


here 


6c=0,  (14) 

^'^^'''  {l<i).  (15) 


Clearly,  by  (11)  and  (12) 
.so  by  (13)  and  (15), 
.so  by  (13)  and  (14), 

By  (13)  and  (16) 


z'[AZi 

2/ri=0,  (l<i).  (16) 

zMZi+i  =  0,         (l<i).  (17) 


2i=ro. 


(18) 


rjzi+i=\r 


=  |r,K  (19) 


Theorem  2.     //  r<?^0  for  0<i<Cni,  then  for  l<j<m,  Zjj^O,  so  that  a,  and  hj  can  he  defined  by  (11) 
d  (15). 

Proof.     By  (13)  and  (16) 

\2i^i?=\ri\'-{-b^\2i\\  (20) 


209431—53- 
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From  this,  our  theorem  follows  by  induction  on  m. 

Clearly,  if  r^^O  for  0<i<m  and  r^=0,  then  by  (9),  Xm=A-'k  and  we  have  solved  (5).     Henceom 
would  never  wish  to  carry  this  procedure  past  the  least  m  for  which  r,n=0. 

Theorem  3.     //  rjj^O/or  0<?'<m,  thenjor  l<j<m,  a;>0. 

For  by  (19)  and  (11). 

But  A  is  positive  definite,  and  2^ 5^0  by  theorem  2,  so  that  2jAzjy>0. 
Theorem  4.     //  ri^O  for  0<i<Cm,  then 

z!rj=0         (l<i<j<m)  (22 

2fr,— |r,_,p         (0<i<.7<m)  (23 

zJ'Ari=2fAzj+r     (l<i<m,     0<j<m,     i^j)  (24 

rlrj=0         (0<i<j<m)  (25 

zfAzj=0         {l<i<j<m-\-l).  (26 

Proo/  6?/  induction  on  m.     First  let  m=l.     Then  we  get  (22)  from  (16),  (23)  from  (19),  (24)  froir 
(18),  (25)  from  (16)  and  (18),  and  (26)  from  (17). 

Assume  the  theorem  for  m,  with  m>\.     Further  assume  that  r,5^0  for  0<i'<m4-l.     By  (12) 

zlri  =  zlri_i  —  aiZlAZi. 

So  b}'   (26),    for    l<7'<w  +  l,     l<h<m-\-l,     i^^h,   we   get    2fri  =  2fr,_i.     If    we    combine    this   witt 
(16),  we  infer  (22)  for  m  +  1.     If  we  combine  this  with  (19),  we  infer  (23)  for  m+l. 
By  (13),  we  get 

zTAzj+i= zfArj + bjzjAzj. 

From  this,  we  get  (24)  for  jn-{-l  by  using  (14)  when  j=0  and  (26)  when  l<j. 

^^^^^^  T  T  r  ' 

If  i<m,   then  rJrn=Q  by   (25),     rJAz^+i  =  zl+iAzi+i  by   (24),    and  0^+,^2,+i  =  O    by    (26).     Also,  i: 
i=m,  then  we  get 

By  (19)  and  (24) 

Then  by  (11),  r^r^+i  =  0.     Thus  we  infer  (25)  for  m+l. 

By   (17)    zl+iAz^+2=0.     By   (13)   and   (26),   we  have  for   l<i<m,   zjAZm+2=zfArr^+i  =  rl+,Az, 
But  by  (12)  and  theorem  3 


So 


A  1.1 


zJAzm+2=-—  r^+ir,+—  rl^,ri_,. 


Then  by  (25),  2fA2„+2=0.     Thus  we  infer  (26)  for  m+l. 

We  note  that  (26)  tells  us  that  the  2's  are  ^-orthogonal,  so  that  we  have  indeed  a  special  case  oi, . 
the  procedures  outlined  above. 

Theorem  5.     If  rtT^Q  for  0<i<m,  the 7i  for  l<j<m,  bj>0. 

Proof.     Bv  (12)  and  (25), 

\r,\'=-a,rfAzj.  (27' 
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en  by  (21)  and  (15), 

bj=P^2  (28) 

From  (28),  we  see  that  if  Tij^O  for  0<'i<m  and  r,„=0,  then  bm=0.     Then  by  (13),  Zm+i  =  0. 
It  is  interesting  that  the  iteration  outhned  above  is  identical  with  that  derived  by  Stiefel  from  an 
irely  different  approach.^ 

By  (12)  and  (13), 

^i+l^f'i-l'TOiZi       ttjAZi 

Zi+i=—aiA2i^il+bi)Zi—bi_iZi_i.  (29) 

Qce  the  z's  are  indeed  constructed  according  to  (8). 

'  Not  only  is  Xn  the  solution  of  (5),  but  one  can  now  readily  solve  Ay=h.  To  do  so,  let  us  seek  c's 
::hsii  A~^h=CiZi-]r  .  .  .  -\-CnZ„.  That  is,  h=CiAzi-\-  .  .  .  -\-CnAZn.  So,  since  the  s's  are  ^-orthogonal, 
l^CjzfAzj.     Thus 

pi  zJAzj 


jtXing  h  be  an  arbitrary  vector,  we  infer 


n       5,  «r 


;=i 


z'^Azj 


One  can  proceed  entirely  by  means  of  the  r's  and  a;'s  without  ever  bringing  in  the  s's  at  all.     For  by 

1)  and  (13), 

I  ri  =  ri_i  — aj/l2i 

'  Azi+i=Ari+biAzi. 

L  r       --a       At    I  ^^  +  <^^+i^%     .  «'  +  i^i  ^ 

Similarly,  from  (10)  and  (13),  we  get 

_  ,  ai-]r(ii+\bi         dt+ibi 

at  at 

Thus  we  can  generate  the  x's  and  r's  by  the  following  recursion. 

Choose  Xq  at  random.     Define  ro=k—Axo,  see  (9).     Define  Xi=Xo+«o^o  (see  (10)  and  (18)).     Then 
»3(9)  ri=r(,—aoArQ.     Hence  we  are  able  to  determine  ao  from  the  condition  rfro=0  (see  (25)). 
One  can  now  proceed  by  setting 

Xi+i=airi-^^iXi-\-  {l—^{)Xi-i. 
\::m  by  (9) 

r<+i=— ai^r<+i8iri+(l— /3<)rt_i. 

iice  one  can  determmc  at  and  /3i  from  the  conditions  rf+,ri=0  and  rf+,rj_i  =  0. 

6.  Other  iterative  schemes.  Since  theorem  1  holds  for  any  B  which  commutes  with  C,  we  can  gen- 
srizc  the  scheme  presented  at  the  end  of  the  last  section  by  using  an  arbitrary  B.  Thou  the  scheme 
ipears  as  follows. 

E.  Stiefel,  see  paper  6,  page  43. 
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We  have  the  symmetric  matrix  A  (the  assumption  of  definiteness  can  now  be  relaxed)  and  the  vectc 
1-.  Choose  B  any  symmetric  positive  definite  matrix  which  commutes  with  A.  For  example,  B  coul 
be  I  or  al+A,  where  a  is  chosen  large  enough  to  insm-e  definiteness.  We  now  undertake  to  generate 
series  of  approximations  x^,  Xi,  .  .  .  to  A~^k,  with  the  property  that  the  residuals  ri=k—Axi  shall  b 
P-orthogonal.  We  merely  follow  the  scheme  outlmed  above.  We  choose  Xo  at  random.  Then  w 
choose  ao.  so  that  Tq  is  5-orthogonal  to  ro—aoAro.  Then  we  define  a;i  =  a;o +0:0^0,  whence  by  (c 
ri  =  ro—aoAro.     In  general,  we  choose  at  and  /S,,  so  that 

-aiAri+^iri+il-l3i)ri-i 
shall  be  i?-orthogonal  to  /■,  and  r,_i.     Then  we  define 


whence  bj^  (9) 


x,+i  =  airi-i-^iXi+(l~l3i)Xi-i, 
rt+i=-aiAri  +  ^ir,+  {l-^i)ri.i. 
It  may  be  more  efficient  to  compute  the  increments  of  the  x's  by  the  formula 

Xi+i—Xi=airi—{l  —  l3i){Xi—Xi-i). 

By  theorem  1,  the  r's  wiU  be  5-orthogonal  in  pairs.  Consequently,  there  must  be  an  m,  0<m<i 
such  that  rjn=0  (and  so  Xm=A~^k)  miless  the  procedure  terminates  sooner  because  our  attempt  to  detei 
mine  at  or  /3j  requires  a  division  by  zero.  In  the  preceding  section,  we  showed  that  if  A  is  positive  definit 
and  B  is  the  identity  matrix,  a  division  by  zero  could  not  be  called  for  before  one  found  an  r,„=0.  V 
the  general  case,  it  is  not  laio^Ti  what  can  happen,  but  it  seems  plausible  that  one  will  have  little  dang( 
in  general  of  encountering  a  division  by  zero. 

7.  Rounding  off  errors.  In  the  scheme  of  section  6,  one  always  computes  the  residuals  directl 
from  A,  k,  and  Xt.  Hence  there  is  no  danger  of  accumulating  a  romid-off  error  in  the  sense  that  on 
thinks  one  has  a  good  Xi,  when,  in  fact,  one  has  a  poor  Xi  because  of  round-off  errors.  One  may  indee 
have  a  poor  Xt  because  of  round-off  errors  (or  even  because  of  actual  mistakes)  when  one  should  have 
good  Xi,  but  the  poorness  of  Xt  will  be  fully  apparent. 

If  there  were  no  round-off  (or  other)  errors,  the  procedure  would  surely  give  A~^k  in  n  or  fewfr 
steps.  However,  because  of  roimd  off,  the  r's  will  not  be  truly  i?-orthogonal,  and  so  the  process  will  nrf 
terminate  in  n  steps.  However,  smce  it  should  terminate  in  n  steps,  it  can  be  expected  to  come  vei-' 
€lose  in  n  steps.  .' 

Note  that  after  the  first  step,  there  is  nothing  to  distinguish  one  step  from  the  next  except  the  siz 
of  the  r's.     If  at  the  end  of  n  steps,  the  r„  is  not  sufficiently  small  (either  from  round  off  or  from  mi-  • 
takes),  there  is  nothing  whatever  to  bring  the  process  to  a  termination.     One  can  simpl}-  let  the  pr 
cedure  continue  until  r^  is  sufficiently  small,  even  if  this  may  in  some  cases  require  a  value  of  m  great 
than  n. 

For  the  procedure  of  section  5,  similar  remarks  hold.     If  xt  is  computed  from  (10)  and  then 
is  computed  from  (9),  leaving  (12)  as  a  check,  one  is  dealing  with  actual  residuals,  and  is  protecte 
from  the  possibility  of  an  error  slipping  in  undetected.     Also,  there  is  absoluteh-  nothing  to  terminal 
the  procedure  at  the  nth  step  if  by  some  mischance  r„  is  not  sufficiently  small. 

It  thus  appears  that  in  actual  computation,  our  schemes  are  iterative  schemes  which  wHl  nc 
ordinarily  terminate  in  any  finite  number  of  steps,  but  which  mnj  be  expected  to  give  a  very  clos- 
approximation  in  n  steps. 

Nothing  is  yet  known  about  the  rapidity  of  convergence  after  n  steps,  but  it  seems  plausible  t 
assume  that  it  is  comparable  to  the  rapidity  of  convergence  during  the  first  n  steps.  Investigation  v 
this  question  is  called  for,  and  a  numerical  testing  of  the  procedures  is  now  in  progress.  Another  que 
tion  of  a  related  sort  is  that  of  what  conditions  on  the  matrix  A  and  the  vector  k  make  it  likely  th{ 
round-off  errors  could  force  r„  to  be  larger  than  allowable,  and  whether,  in  such  case,  it  is  more  profitab : 
to  let  the  iteration  continue  or  to  make  a  new  start  with  the  Xn  which  has  been  obtained.  1 
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8.  Some  Problems  in  Aerodvnamies  and  Strnctnral 
Engineering  Related  lo  Eigenvalnes 

R.  A.  Frazer  ' 

Scope  of  the  Paper 

The  problems  of  aerodynamics  and  structural  engineering  with  which  this  paper  is  concerned  relate 
/moving  fluids  and  elastic  bodies  and  therefore  to  continuous  systems.  The  anal^^sis  of  the  effects 
)  small  disturbances  of  such  systems  leads  to  sets  of  simultaneous  linear  and  homogeneous  equations 
^icli  contain  one  or  more  parameters.  Without  attempting  a  rigorous  or  general  definition,  I  shall 
lerpret  the  term  "eigenvalue"  to  mean  anj'  value  of  a  t^^pical  parameter  for  which  such  sets  of  equations 
ive  a  definite  nontrivial  solution.  Other  terms  in  current  usage  are  "proper  value"  and  ''characteristic 
imber". 

It  will  be  obvious  that,  in  general,  the  only  practicable  method  of  solution  of  these  problems 
^  )y  means  of  approximations.  Even  so,  an  approximate  treatment  of  the  continuous  systems  concerned 
^uld  present  prohibitive  difficulties  were  it  not  for  one  circmnstance.  This  is  bound  up  with  the 
uctical  uses  to  which  the  solutions  are  finally  to  be  put.  An  aircraft  designer  concerned  with  these 
))blems  is  not  interested  to  know  how  his  structure  woidd  behave  in  all  conceivable  circumstances; 
I  only  wants  to  be  assured  that  it  will  be  safe  in  actual  use.  To  the  mathematician,  this  limitation 
rans  that  only  a  few  of  the  infinite  number  of  possible  sets  of  eigenvalues  need  be  determined,  and 
k  takes  advantage  of  this  fact  to  simplify  his  analysis.  His  procedure,  in  brief,  is  to  replace  the  con- 
mous  system  by  a  finite  one  chosen  in  such  a  way  that  the  two  systems  have  (approximately  at  least) 
?  appropriate  band  of  eigenvalues  in  common.  To  obtain  successful  approximating  systems,  he  has, 
.  present,  to  rely  heavily  on  physical  intuition.  A  more  abstract  treatment  may  be  found  possible 
'en  the  general  principles  underh'ing  the  approximations  are  better  understood.  But,  for  the  purposes 
I  this  paper,  many  references  to  the  physical  background  will  be  necessary  in  order  to  provide  a  clear 
■cunt. 

I  Approximating  Systems  in  Flutter  Analysis 

I 

'  One  problem  of  great  practical  importance  in  relation  to  airplane  design  is  the  prevention  of  un- 
jble  structural  oscillations  of  the  type  known  as  flutter.  With  a  defective  design,  it  is  possible  for 
i  airplane  in  steady  flight  at  any  given  altitude  to  develop  flutter  for  certain  ranges  of  the  flight  speed, 
lie  analysis  is  generally  restricted  to  a  calculation  of  the  "critical  speeds",  which  define  the  end  points 
[the  speed  ranges  for  flutter  and  which  therefore  lead  to  simple  harmonic  oscillations  of  the  structure, 
iknowledge  of  the  lowest  critical  speed  is  usually  sufficient  for  practical  purposes,  as  this  decides  the 
!;hest  safe  speed  for  the  airplane  at  the  given  height. 
!    The  essential  task  confronting  the  flutter  analyst,  then,  is  not  to  describe  fufiy  the  flutter  properties 

a  given  airplane,  })ut  merely  to  locate  its  lowest  critical  speed.  Moreover,  great  precision  in  the 
termination  of  this  speed  will  be  unnecessary  because  a  factor  of  safety  will  be  allowed.  He  is  there- 
•e  entitled  to  substitute  any  mechanically  different  and  simpler  system  which  possesses  approximately 
s  same  lowest  critical  speed.     A  replacement  of  this  nature,  to  give  the  same  accuracy,  could  o])viously 

done  in  man}'  different  ways:  and  the  optimum  way  in  any  given  case  would  dei)end  partly  on  the 
lount  of  calculation  involved  and  partly  on  the  ease  of  formulation.  It  is  probable  that  there  is  no 
e  procedure  that  is  optimum  for  all  classes  of  flutter  problems.     At  present  there  is  no  certaint  \'  even 

to  the  best  methods  to  be  used  for  i)arti<'ul}ir   classes. 

'  National  Physical  Laborator>'  at  Teddington,  Middlesex,  Kngland.    The  pai)or  is  puMishcd  by  pcnnission  of  the  Director  of  the  I/ahoratory. 
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The  first  step  in  a  flutter  analysis  is  to  decide  on  the  type  of  the  approximating  system  and  on  the 
number  of  the  freedoms.  A  description  of  the  most  generally  favoured  system  will  serve  as  an  illustra- 
tion; some  others  will  be  mentioned  later.  The  orthodox  system  is  a  semirigid  copy  of  the  airplane,  or 
of  the  aircraft  component  concerned.  This  system  has  the  same  external  appearance  and  the  same  mass 
distribution  as  the  original  structure,  but  is  a  mechanism  possessing  chosen  deformation  modes  in  its 
several  freedoms.  It  is  usual  to  base  these  modes  and  freedoms  on  the  lower  order  natural  oscillations 
of  the  structure,  and  these  are  sometimes  calculated  and  sometimes  determined  experimentally  by  res- 
onance tests.  This  procedure  is  probably  the  safest,  but  satisfactory  results  have  also  been  obtained 
from  modes  appropriate  to  static  loading,  or  from  modes  represented  by  polynomials  or  other  simple 
functions  of  position.  The  number  of  modes  and  freedoms  to  be  included  in  a  flutter  calculation  is 
usually  a  matter  for  judgment.  With  complicated  problems,  such  as  those  involving  possible  interactions 
between  the  wings,  tail  organs  and  fuselage,  a  decision  may  be  very  difficult.  In  doubtful  cases,  it  may 
well  be  necessary  to  introduce  more  and  more  freedoms  until  results  have  settled  down.  The  greatest 
number  of  freedoms  used  for  a  single  systematic  flutter  analysis  at  Teddmgton  has  been  twelve.  The 
investigation  related  to  the  natural  oscillations  and  critical  flutter  speeds  of  an  exceptionally  large  air- 
plane [1],  and  the  numerical  work,  which  was  mainly  carried  out  by  two  computers  using  electrical 
machines,  occupied  about  two  years. 

Once  the  approximating  system  has  been  chosen,  the  next  step  is  to  obtain  the  expressions  for  the 
aerodynamic  loads  on  the  oscillating  system.  A  knowledge  of  these  aerodjmamic  coefficients  is  vital  to 
the  whole  analysis,  and  much  effort  is  being  devoted  to  their  determination  both  theoretically  and  ex- 
perimentally. However,  this  highly  important  and  highly  specialised  branch  of  aerodynamics  can  only 
be  mentioned  here  in  passing.  To  complete  the  analytic  formulation,  the  appropriate  nondimensional 
aerodynamic,  inertial,  and  elastic  coefficients  are  introduced  into  the  dynamical  equations.  If  the  as- 
sumed system  has  n  degrees  of  freedom,  two  conjugate  complex  sets  of  n  simultaneous  algebraic  equations 
are  finally  obtained,  each  set  being  linear  and  homogeneous  in  n  complex  unknowns.  These  unknowns 
determine  the  amplitude  and  phase  relations  between  the  oscillatory  movements  in  the  different  freedoms. 

The  parameters  dependent  on  the  flight  conditions  which  can  be  present  in  the  equations  are  the 
speed  parameter  (which  is  inversely  proportional  to  the  square  of  the  flight  speed) ,  the  frequency  param- 
eter (which  is  proportional  to  the  ratio  of  frequency  to  flight  speed) ,  the  Mach  number  (ratio  of  flight 
speed  to  speed  of  sound  in  the  free  stream),  and  the  air  density  ratio  (flying  height  to  ground  level). 
All  these  must  take  real  positive  eigenvalues,  and  some  are  interrelated.  Other  parameters  will  also  be 
present  if  any  of  the  data  connected  with  the  actual  design  of  the  structure  are  variable. 
The  critical  equations  can  be  expressed  by  matrices  as 

[y±iv]{x±iy}=0,  (1) 

where  u  and  v  denote  real  square  matrices  of  order  n,x±iy  denote  the  column  vectors  of  the  unknowns, 
and  ^  denotes  the  imaginary  unit.     Then  the  compatibility  conditions  in  determinantal  form  are 

Ai=  li^+n'l  =0     with     A2=\u—iv\=0.  (2) 

The  matrices  u  and  v  will  in  general  be  asymmetric  owing  to  the  asymmetry  of  the  aerodynamic 
coupling  coefficients.  Their  elements  will  be  linear  in  the  speed  parameter  but  transcendental  in  the 
frequency  parameter  and  Mach  number.  Thus,  in  general,  the  only  practicable  basis  of  calculation  is 
to  introduce  trial  values  for  the  frequency  parameter  and  the  Mach  number  until  both  conditions  (2) 
are  satisfied  by  a  corresponding  real  positive  value  for  the  speed  parameter.  This  process  can  sometimes 
be  assisted  by  a  judicious  use  of  any  structural  design  parameters  present.  It  is  also  greatly  simplified 
when  the  effects  of  compressibility  can  be  ignored  and  the  Mach  number  is  thus  absent. 

In  British  practice,  the  transcendental  functions  of  the  frequency  parameter  in  the  aerodynamic 
coefficients  relevant  to  normal  flight  speeds  are  sometimes  replaced  by  quadratic  approximations. 
The  expanded  forms  of  the  determinantal  conditions  (2)  can  then  be  interpreted  graphically  as  two 
polynomial  curves  of  degrees  n  and  n—1  in  the  plane  of  the  speed  and  frequency  parameters."^  Corre- 
sponding pairs  of  practical  eigenvalues  are  given  by  those  intersections  of  the  curves  that  lie  in  the  first 
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drant.  Several  methods  of  calculation  based  on  interpolation  formulas  for  the  polynomials  have 
ei  suggested  [2,  3].  In  one  of  these,  an  interpolation  framework  of  straight  lines  ii  =  0,  L2=0,  Lz=0, 
a  so  on,  is  adopted,  no  two  lines  being  parallel  and  no  three  concurrent.  The  polynomials  are  then 
icmded  in  bivariate  partial  fractions  in  which  the  denominators  are  the  products  of  the  linear  expres- 
cs  Li,  L2,  L3,  etc.,  taken  m  pairs.  The  numerators  of  the  fractions  are  the  values  of  the  pohniomials 
b  lie  line  intersections,  and  these  are  calculated  directly  as  numerical  determinants  Aj  and  A2.  With 
1  use  of  a  standard  line  framework  -and  prepared  double-entry  tables  for  the  fractions,  the  search  for 
1  intersections  of  the  curves  becomes  a  routine  procedure  that  might  perhaps  be  suitable  for  Hollerith 
[•  thor  automatic  computing  equipment.  A  disadvantage  of  these  methods  is  that  with  many  degrees 
[ '.eedom  the}^  demand  ver\^  high  accuracy. 
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Figure  8.1.     Critical  speeds  V  and  frequencies  F  for  model  wing. 

'  An  example  of  a  simple  flutter  diagram  is  given  by  figure  8.1.  The  diagram  compares  the  lowest 
•  ical  speeds  observed  for  a  model  cantilever  wing  in  -a  wind  tunnel  [4]  with  results  calculated  by  the 
hariate  partial  fractions  method.  The  model  w^ing  carried  a  variable  concentrated  mass  M representing 
ing  engine,  and  the  approximating  system  adopted  had  four  freedoms  and  simple  pohmomial  distortion 
'des.  Standard  air  density  was  assumed,  and  the  three  parameters  present  in  the  analysis  were  the 
i  ical  speed,  the  critical  frequency,  and  the  engine  mass.  The  modes  of  oscillation  and  the  frequencies 
"e  markedly  different  for  the  two  branches  AB,  BC  of  the  critical-speed  curve.  For  AB  the  wing 
ition  containing  the  engine  mass  showed  little  movement,  but  with  BC  it  pitched  vigorously  and  at  a 
liceably  higher  frequency.  When  the  engine  mass  was  adjusted  to  accord  with  the  nodal  point  B, 
|ier  type  of  oscillation  could  be  obtained  according  to  the  way  in  which  the  wing  was  disturbed. 
I  An  example  of  an  unorthodox  approximating  system  in  flutter  analysis  is  provided  by  a  segmented 
lirigid  wing.  Systems  of  this  type  have  been  used  [5]  in  connection  with  the  flutter  of  solid  blades 
1  airscrews.  The  elastic  \\'ing  is  regarded  as  segmented  by  crosscuts  into  either  rigid  slices,  or  into 
es  that  are  simply  semirigid  so  that  continuity  of  the  flexural  and  torsional  displacements  is  preserved 
ur  the  span.  The  slices  are  interconnected  by  springs  that  are  so  adjusted  that  the  fundamental 
illations  of  the  actual  and  approximate  systems  agree.  Critical  speeds  predicted  by  this  method 
1  b}'  the  standard  semirigid  method  have  been  found  to  agree  well. 

Approximations  by  Lumped  Coefficients 

Another  possible  means  of  representing  a  continuous  system  approximately  is  b}'  a  system  of 
Crete  massive  particles  with  elastic  connections.     The  theory  of  the  relationships  between  continuous 
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and  discrete  systems  is,  of  course,  a  very  wide  subject,  which  has  been  treated  on  a  general  basis  b 
matrices  in  recent  papers  by  several  \vriters  [6,  7],  both  in  relation  to  mechanics  and  electrodynamics 

The  form  of  approximation  relevant  to  the  present  context  is  the  simple  process  that  is  sometime 
used  to  obtain  estimates  of  the  natural  frequencies  of  elastic  beams  and  shafts.  It  is  often  referred  t 
by  engineers  as  the  method  of  "lumped  masses".  The  application  of  this  principle  to  flutter  analysi 
has  hitherto  been  very  limited.  Partial  use  of  it  is  made  when  special  parts  of  an  aircraft  structur( 
such  as  engine  nacelles,  wing  fuel  tanks  or  fuselage  components  are  replaced  by  one  or  more  massiv 
particles.  But  fully  discrete  systems,  in  which  not  only  the  masses  or  inertial  coefficients,  but  also  th 
aerod}Tiamic  coefficients  are  "lumped",  have  not  so  far  been  systematically  studied.  A  simple  exampl 
of  such  a  system  \n\\  be  given  later. 

Some  instructive  conclusions  on  the  effects  of  mass  lumping  can  be  drawn  from  oscillating  tensione.. 
strings.  The  natural  frequencies  of  transverse  oscillation  of  stretched  continuous  strings  and  approxi 
mating  discrete  strings  have  been  compared  by  Den  Hartog  [8].  He  assumes  that  all  the  strings  havi 
the  same  length  and  same  total  mass  n,  and  that  each  discrete  string  is  massless  but  carries  equal  massiv 
particles  at  equal  distances  apart  across  the  span.  As  the  total  string  mass  is  kept  constant,  the  mas 
values  for  the  individual  particles  will  depend  on  whether  or  not  one  of  the  particles  is  also  allowed  t 
be  present  at  each  fixed  end.  Results  are  given  by  Den  Hartog  for  both  cases.  When  the  chain  o 
particles  stops  short  of  the  ends,  the  approximate  fundamental  frequencies  are  too  low,  but  when  i 
includes  the  ends  the  frequencies  are  too  liigh.  In  either  case,  the  true  frequency  value  is  approachei 
only  slowly  as  the  number  of  the  particles  is  increased.  It  can  however,  be  objected  that  neither  o 
these  arrangements  provides  a  really  fair  representation  of  a  continuous  string.  If  such  a  string  is  t' 
be  segmented  into  links,  the  simplest  procedure  evidently  is  to  assign  one-half  of  the  mass  of  eachlin] 
to  the  end  points  of  that  link.  So,  if  in  particular  the  string  is  uniform  and  the  links  are  all  equal,  th 
discrete  system  carries  equal  massive  particles  within  its  span,  but  only  half  masses  at  its  two  ends 
On  this  supposition  the  method  is,  in  fact,  more  promising,  as  will  be  seen  from  figure  8.2. 
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Figure  8.2.     Lowest  frequencies  of  discrete  strings. 

When  half  masses  are  present  at  the  ends,  the  natm-al  frequencies  of  any  given  order  corresponding 
to  the  discrete  and  the  continuous  strings  are  in  the  ratio 
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'here  n  denotes  the  number  of  equal  Hnks  in  the  approximate  sj^stem  and  j  (which  ranges  from  1  to 
—  1)  denotes  the  number  of  antinodes  in  the  oscillation  wave  form.  With  the  fundamental  mode 
\j=\)  the  error  in  frequency  is  10  percent  when  only  two  links  are  used,  and  is  less  than  5  percent  with 
laree  links.  In  the  case  of  the  first  overtone,  four  links  and  six  links  are  needed,  respectively,  for  the 
ime  accuracies. 

I  In  connection  with  this  method  of  approximation,  it  is  to  be  noted  that  the  differences  between 
le  displacement  modes  can  be  pronounced,  even  when  the  frequencies  agree  well.  With  the  continuous 
:ring,  the  wave  forms  of  displacement  are  sine  ciu-ves.  With  the  discrete  strings,  the  modes  are  defined 
\  the  first  place  by  the  displacements  of  the  lumped  particles  and  in  the  second  place  by  the  displacements 
f  the  links.  The  displacements  of  the  particles  accord  with  sine  curves,  but  the  links  are  massless  and 
:3main  straight.  So  the  complete  modes  are  composed  of  blunted  sine  curves,  the  blunting  increasing 
[3  the  number  of  links  is  reduced.  If  modes  free  from  these  imperfections  are  required,  they  must 
e  calculated  for  the  continuous  string  with  the  use  of  the  approximate  frequencies. 
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Figure  8.3.     Symmetric  oscillations  of  loaded  discrete  strings. 

,  Figure  8.3  gives  some  comparisons  of  natural  frequencies  for  continuous  and  discrete  strings  of 
Iqual  length,  I,  and  equal  mass,  n,  when  an  extra  concentrated  mass,  m,  is  carried  at  the  centre.  In 
ach  case  the  string  mass  is  made  up  of  equal  equidistant  particles  except  that  only  half-particles  are 
;laced  at  the  ends.  The  ordinate  in  the  diagram  is  the  ratio  m/n  (extra  mass  to  string  mass),  and  the 
bscissa  is  proportional  to  the  natural  frequency.  For  simplicity,  results  are  only  shown  for  the  funda- 
iiental  and  first  overtone  symmetric  oscillations.  It  is  seen  that  the  discrete  strings  all  provide  good 
rst  approximations  for  the  fundamental  frequency,  and  that  fair  approximations  are  obtained  in  the 
ase  of  the  first  overtone  even  when. only  few  particles  are  used. 

I  A  similar  method  of  approximation  is  possible  for  more  general  systems  containing  two  or  more 
'snsioncd  strings  cross  connected  by  sets  of  rigid  rods  or  ribs.  A  simple  analogue  to  a  uniform  rectangular 
irplane  wing  with  its  two  spars  held  at  both  ends  can  be  (k^vised  by  substituting  two  tensionod  massless 
itrings  for  the  wing  spars  and  cross  connecting  them  by  a  close  pack  of  separate  thin  ribs.  If  the  number 
f  rib  strips  is  very  great,  this  articulated  imitation  wing  is  effectively  continuous.  Any  natural  trans- 
■erse  oscillation  will,  in  general,  consist  of  a  torsional  movement  about  some  axis  parallel  to  the  span, 
nd  the  twist  will  be  distributed  sinusoidally  across  the  span. 

To  obtain  an  approximating  system,  it  is  only  necessary  to  segment  the  strings  into  equal  (inito 
nks  and  to  treat  the  rib  masses  as  though  they  wore  concentrated  at  the  ends  of  the  links.    The  wing 
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cover  can  be  pictured  as  being  carried  by  the  ends  of  the  Unks  in  sohdified  pieces,  the  divisions  between 
the  pieces  being  central  in  the  Hnks.  Thus,  when  the  system  oscillates,  each  discrete  strip  of  wing  moves 
with  its  bridging  rib  two-dimensionally,  pitching  angularly  and  translating  vertically.  The  two  half 
strips  attached  to  the  fixed  ends  will,  of  course,  remain  idle.  As  might  be  expected,  corresponding  natural 
frequencies  of  the  approximating  and  continuous  wing  systems  are  here  in  the  same  ratio  as  that  found 
for  a  single  string. 

If  the  continuous  wing  had  unfavorable  inertias  and  elastic  stiffnesses  and  were  exposed  to  steady 
winds,  it  would  be  liable  to  flutter  in  modes  involving  coupled  bending  and  torsion.  This  simple  flutter 
problem  can  be  solved  formally  and  in  finite  terms,  on  the  assumption  that  aerodynamic  coefficients 
appropriate  to  two-dimensional  air  flow  are  applicable  over  the  wing  surface.  The  problem  is  also 
soluble  for  the  approximating  systems,  provided  the  coefficients  used  to  evaluate  the  aerodynamic  forces 
over  the  discrete  strips  of  wing  are  again  applicable  to  two-dimensional  air  flow.  The  procedure  in 
the  approximate  calculation,  therefore,  is,  effectively,  to  lump  not  only  the  masses  but  also  the  aerody- 
namic coefficients  at  the  ends  of  the  links.  It  is  found  that  for  incompressible  flow  corresponding  critical 
speeds  for  the  approximate  and  continuous  wings  are  again  in  the  same  ratio  as  that  applicable  for  natural 
frequencies.  In  particular,  the  lowest  critical  speed  (namely,  that  appropriate  to  flutter  in  the  first  mode) 
svill  be  10  percent  in  error  with  only  two  links  or  one  active  strip,  and  w^ill  be  under  5  percent  in  error 
with  three  links  or  two  active  strips.  These  approximations  would  be  too  low  and  would  thus  err  on  the 
safe  side  if  practical  applications  were  in  view.  All  the  wings  will  show  flutter  in  any  given  mode  for  the 
same  value  of  the  frequency  parameter.  These  conclusions  can  be  drawn  without  any  specific  knowledge 
of  the  way  in  which  the  two-dimensional  aerod\Tiamic  coefficients  depend  functionally  on  the  frequency 
param.eter.    Moreover,  they  would  not  be  influenced  very  materially  by  the  effects  of  compressibility. 

The  example  just  given  is  crude,  and  it  may  weU  be  misleading.  However,  there  is  some  other 
evidence  that  approximations  of  this  type  deserve  more  attention.  The  well-known  method  due  to 
Theodorsen  and  Garrick  [9]  for  the  prediction  of  the  flutter  of  cantilever  wings  is  effectively  dependent 
on  lumped  coefficients,  and  it  gives  results  that  compare  w^ell  with  those  obtained  by  more  elaborate 
methods  [10].  In  general,  a  lumped  coefficient  method  of  flutter  prediction  would  have  several  advan- 
tages and  one  main  disadvantage.  On  the  one  hand,  it  would  require  no  previous  knowledge  of  the 
distortion  modes  and  would  be  relatively  easy  to  formulate.  On  the  other  hand,  if  the  convergence  were 
slow,  the  method  would  prove  expensive  in  regard  to  the  number  of  freedoms,  because  each  additional 
Imk  introduced  would  require  a  corresponding  increased  nimiber  of  freedoms.  More  light  on  this  question 
and  on  the  general  flutter  properties  of  actual  wings  may  be  obtained  from  the  study  of  further  simple 
articulated  w'mg  systems,  such  as  those  carrying  extra  concentrations  of  mass  or  having  more  general 
plan  forms.  Another  obscure  and  vital  question  is  the  possible  application  oi  lumped  coefficients  to 
control  surfaces. 

Lumped  Coefficients  in  Structural  Engineering 

Before  some  of  the  other  problems  of  fluid  dynamics  are  mentioned,  it  may  be  of  interest  to  exemplify 
the  use  of  lumped  coefficients  in  structural  engineering.  The  natural  oscfllations  of  framed  structures 
were  first  treated  in  detail  by  Reissner  [11].  He  regards  the  structure  as  a  dynamical  sj^stem  composed 
of  interconnected  massless  elastic  bars  with  compensating  lumped  masses  placed  at  all  the  joints.  How- 
ever, the  complications  connected  with  the  many  degrees  of  freedom  and  with  the  solution  of  the  fre- 
quency equation  precludes  the  general  use  of  his  method  for  extensive  structures  such  as  bridges. 

A  theoretical  study  of  the  stiffness  and  frequency  properties  of  bridge  box  structures  [12]  has  recently 
been  carried  out  at  Teddington  as  part  of  an  investigation  on  the  aerodynamic  oscillations  of  suspension 
bridges,  and  some  of  the  results  obtained  throw  further  light  on  the  influence  of  Imnped  masses  on  natural 
frequencies.  Before  the  frequency  diagrams  are  shown,  some  explanations  on  the  basis  of  calculation 
will  be  necessary. 

A  suspension-bridge  box  structure  consists  of  a  number  of  identical  framed  box-form  bavs  connected 
in  series  by  cross  girders.  These  cross  girders  play,  effectively,  the  part  of  transverse"^  diaphragms 
separating  the  bays.  In  practice  the  weight  of  the  structure  is  taken  by  the  cables  through  vertical 
ropes,  or  suspenders,  attached  to  the  cross  girders.     However,  the  i)resent  method  of  analysis  is  concerned 
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Dlelv  with  the  frequency  properties  of  the  suspended  structure  itself  in  the  absence  of  suspenders  and  a 
ravity  field.  As  an  approximation,  the  masses  of  the  structure  are  assumed  to  be  lumped,  not  at  the 
idividual  joints  as  in  Reissner's  theory,  but  at  cross-girder  positions;  and  the  inertial  condition  is  de- 
3ribed  as  "close"  or  "open"  according  as  all  the  cross  girders  or  merely  a  selected  equidistant  set  are 
lass-loaded.  The  treatment  thus  resembles  that  already  explained  for  strings,  except  that  with  the 
ridge  the  links  consist  of  lengths  of  the  elastic  structm-e  which  are  necessarily  multiples  of  the  span  of  a 
ingle  baj^. 

When  the  structure  bends  vertically  any  cross  girder  is  assumed  to  displace  as  a  rigid  body,  and  it  is 
llowed  two  degrees  of  freedom — vertical  translation  and  angular  flexural  displacement.  With  open 
)ading  the  displacements  of  the  mass-loaded  cross  girders  are  regarded  as  primar}^.  The  other  girders 
e  within  the  lengths  of  structure  that  form  the  links,  and  their  displacements  are  therefore  determined 
y  the  primary  displacements  and  the  elastic  stiffness  constants  of  the  system.  "When  the  structure 
scillates,  the  links  do  not  remain  straight  as  for  strings,  but  curve. 
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Figure  8.4.     Bending  oscillations  of  a  bridge  box  structure. 


j  Figure  8.4,  which  is  plotted  similarly  to  figure  8.3,  illustrates  the  influence  of  open  loading  on  the 
lymmetric  flexural  frequencies  of  a  box  structure  of  total  mass  n,  with  an  extra  concentrated  mass  m 
It  midspan.  The  datum  system  is  the  close-loaded  structure  containing  six  simple  bays  in  each  half-span, 
'nd  the  most  open  one  contains  merely  one  link  in  each  half-span.  Only  the  three  lowest  order  oscilla- 
lons  are  covered  by  the  diagram.  The  system  with  only  one  link  in  each  half-span  yields  a  single 
jscillation,  but  it  nevertheless  provides  a  good  approximation  for  the  fundamental  frequency. 
'  The  corresponding  results  for  symmetric  torsional  oscillations  are  shown  in  figure  8.5.  Here  the 
Issumed  freedoms  are  the  twist  of  the  structure  at  the  cross-girder  positions,  and  the  twisting  or  warping 
jisplacements  of  the  cross-girder  planes. 

,  Both  of  the  two  problems  that  have  been  exemplified  are  fomially  soluble,  on  the  assumptions  made, 
'i  finite  tenns.  The  dynamical  equations  determinuig  the  movements  of  the  mass-loaded  cross  girders 
jnder  the  elastic  forces  due  to  the  structural  links  yield  a  set  of  simultaneous  recurrence  relations  connect- 
ig  the  displacement  amplitudes  of  any  three  such  consecutive  girders  and  containing  the  squared  fre- 
uency  as  a  parameter.     A  treatment  of  these  relations  by  the  methods  available  for  shnultaneous  difl'er- 
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ence  equations  leads  to  a  formal  solution.  An  alternative  method  is  to  express  the  equations  m  bulk  by 
matrices  and  so  to  relate  the  problem  to  the  determination  of  latent  roots.  Problems  of  this  type  yield 
partitioned  matrices  composed  of  alternant  submatrices  and  irregular  rows  dependent  on  the  extra 
mass-concentrations  present.  The  examples  given  are,  of  course,  only  special  cases  of  more  general 
repetitive  systems  or  dynamic  chains,  composed  of  any  number  of  identical  massive  semirigid  bodies 
or  carriers  joined  together  in  series  by  identical  massless  elastic  connectmg  sytems. 
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Figure  8.5.      Torsional  oscillations  of  a  bridge  box  structure. 


The  results  for  the  box  structure  suggest  that  it  may  also  be  possible  to  obtain  satisfactory  estimate^ 
of  the  lower  natural  frequencies  of  complete  suspension  bridges  by  means  of  lumped  coefficients.  How 
ever,  no  definite  conclusions  on  this  important  question  can  be  drawn  at  present.  A  knowledge  of  thes 
frequencies  plays  a  vital  part  in  the  prediction  of  the  critical  speeds  at  which  oscillations  of  the  bridge 
are  set  up  due  to  wind.  With  a  long  bridge  the  normal  procedure  in  frequency  calculations  is  to  regan 
the  framed  suspended  structure  in  each  span  of  the  bridge  as  effectively  a  continuous  elastic  beam  and  t( 
replace  the  discrete  sets  of  vertical  suspenders  by  continuous  sheets  of  suspenders.  The  displacement 
modes  of  the  structure  and  cables  are  then  given  b}'^  simultaneous  linear  differential  equations  containing 
the  squared  frequency  as  a  parameter  and  coefficients  which  are  variable  with  distance  along  the  span 
The  analytical  difficulties  presented  b}"  these  equations  are  usually  countered  b\^  further  simplifying  as- 
sumptions. It  is  seen,  then,  that  the  conventional  calculation  smoothes  out  the  panneling  of  the  actua 
hridge  design  and  regards  the  distribution  of  the  loading  to  be  continuous.  The  proposed  alternative 
treatment  is,  in  effect,  to  reduce  the  number  of  effective  load-bearing  cross  girders  and  suspenders  to  i 
few  only.  However,  this  method  requires  careful  investigation  because  with  sj'^stems  such  as  suspensioi 
bridges,  which  are  controlled  partly  by  gravity,  mass  lumping  is  accompanied  by  a  corresponding  lumping 
of  the  gravitational  stiffnesses.  The  variation  of  these  stiffnesses  with  the  distance  along  the  span  ii 
another  complication.  It  has  been  suggested  b}^  Pugsley  [13]  that  the  use  of  onh^  a  few  flexibility  coefH 
cients,  determined  for  the  cables  and  suspended  structure  separately,  might  prove  adequate  for  a  siis 
pension-bridge  analysis. 


Stability  of  Fluid  Flow 

To  conclude  this  short  paper,  a  few  remarks  may  be  added  ou  one  or  two  of  the  other  aerodynamic 
"oblems  that  are  related  to  eigenvalues  or  simultaneous  equations. 

One  subject  that  has  tried  the  ingenuity  of  mathematicians  for  many  ye&Ts  is  the  stability  of  the 
Dw  of  an  incompressible  viscous  fluid.  The  first  problem  of  this  tj-pe  to  be  attacked  successfully  con- 
'med  the  flow  between  two  coaxial  circular  cylinders  in  steady  rotation.  It  has  been  shown  by  Taylor 
4],  both  experimental^  and  theoretically,  that  when  the  rates  of  rotation  and  the  radii  are  appropriately 
•lated  the  simple  two-dimensional  rotary  flow  between  the  cylinders  becomes  divergently  unstable. 
I  the  critical,  or  neutral,  state  the  flow  pattern  assumes  a  banded  appearance  due  to  alternations  of 
Dsitive  and  negative  vortices  spaced  regularly  along  the  axis.  Taylor  bases  his  analysis  on  Fourier- 
essel  expansions,  and  the  eigenvalues  he  determines  are  the  widths  of  the  vortex  bands.  He  calculates 
lem  by  the  approximate  solution  of  a  determinantal  equation  of  infinite  order. 

An  alternative  treatment  of  the  same  problem  proposed  by  Jones  [15]  shows  the  possibilities  of 
mple  methods  of  approximation.  He  represents  the  velocity  components  of  the  flow  by  polynomials 
lat  satisfy  all  the  boundary  conditions  and  contain  sets  of  free  constants.  The  errors,  when  these 
>:pressions  are  substituted  in  the  flow  equations,  are  made  zero  for  a  certain,  number  of  equidistant 
idii  and  the  disposable  constants  are  eliminated.  This  simple  collocation  process  leads  to  a  determi- 
antal  equation  of  finite  order  for  the  critical  wave  length,  the  order  depending  on  the  number  of  constants 
sed.     In  the  cases  tried  out  numerically,  good  agreement  with  Taylor's  results  was  obtained. 

With  more  general  boundary  shapes,  approximations  must  also  be  used  to  determine  the  steady 
indisturbed  flow,  and  this  in  itself  presents  a  severe  problem.  Recent  papers  by  Chi-Teh  Wang  and 
Irodsky  [16,  17]  have  sho^^^l  that  solutions  of  fluid-flow  problems  can  be  obtained  successfully  by  the 
iayleigh-Ritz  procedure  or  by  Galerkin's  method.  However,  a  direct  application  of  such  methods  to 
lie  complete  hydro  dynamical  equations  leads  to  S3^stems  of  nonlinear  simultaneous  equations  for  the 
ree  constants.  To  overcome  this  difficulty  it  is  usual  to  approach  the  solution  by  successive  linear 
pproximations. 

One  such  method  that  has  been  suggested  recently  [18]  for  incompressible  flow  assumes  that  the 
•roblem  has  been  solved  for  a  perfect  fluid,  so  that  the  velocity,  g,  with  which  the  fluid  slips  tangentially 
t  each  point  of  the  solid  boundary  is  known.  The  body  is  then  imagined  to  be  covered  with  a  thin 
xtensible  skin  which  is  moved  continuously  along  the  surface  with  speed  {l—\)qsit  each  point,  where  X 
^  a  variable  parameter.  If  viscosity  is  now  introduced,  a  whole  class  of  problems  is  obtained,  all  relat- 
;ig  to  the  same  boundary  shape  and  the  same  viscous  stream  and  differing  only  as  regards  the  condition 
f  slip  at  the  boundar^^  The  problems  range  from  X=0  (when  full  slip  is  allowed)  to  X=l  when  the  slip 
5  completely  annihilated.  If  the  solution  for  the  class  is  assumed  to  be  expansible  in  ascending  powers 
if  X,  the  terms  are  given  in  succession  by  a  sequence  of  linear  differential  equations.  The  first  term  of 
he  expansion  is  the  known  solution  for  perfect  flow,  and  the  second  term  is  given  by  the  set  of  differen- 
'■al  equations  treated  by  numerical  methods  b}-  Southwell  and  Squire  [19].  A  build-up  of  an  approxi- 
nate  solution  b}"  the  methods  of  Galerkin  or  collocation  would  be  attractive  from  the  standpoint  of 
generality,  but  would  require  equipment  for  the  solution  of  linear  numerical  equations  of  very  large 
[►rder.  This  method  has  not  yd  been  put  to  a  test,  and  its  eft'ectiveness  can  at  present  only  be  conjectured. 
The  study  of  disturbed  unsteady  flow  presents  unusual  difficulties  and  is  an  almost  unexplored 
leld.  Perhaps  the  simplest  illustration  of  a  problem  of  this  type  is  shown  by  the  behavior  of  fluid 
contained  within  a  long  rotating  glass  tube.  If,  when  both  the  tube  and  the  fluid  are  in  steady  rotation 
ogether,  the  tube  is  abruptly  arrested,  it  will  be  found  that  the  fluid  motion  does  not  decay  two-dimen- 
lionally  but  that  it  develops  vortex  bands  similar  to  those  observed  by  Tajdor  with  coaxial  cylinders, 
rhe  peculiarity  of  this  problem  is  that  the  deviant  flow  due  to  small  disturbances  becomes  significant 
n  relation  to  the  undisturbed  flow,  although  both  motions  eventually  die  away.  No  definition  of 
nstability  appears  to  exist  which  covers  phenomenon  of  this  kind. 

The  subject  of  disturbed  unsteady  motion  has  wide  applications  and  includes  for  instance  the 
stability  analysis  of  steady  aerodynamic  oscillations.  Such  problems  give  rise  to  linear  differential 
}quations  with  coefficients  which  are  variable  with  time.     If  average  time  values  are  adopted  over 
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successive  small  intervals,  the  complete  motion  can  be  pictured  as  the  limit  of  a  succession  of  disturbec 
steady  motions,  each  associated  with  an  appropriate  set  of  eigenvalues. 

There  are,  of  course,  many  other  interesting  aspects  of  aerodynamics  and  structural  engineering 
that  are  related  to  eigenvalues,  but  they  are  beyond  the  scope  of  a  short  survey.     To  close  this  paper, 
can  therefore  only  regret  both  its  omissions  and  its  treatment,  which  has  inevitably  laid  greater  emphasii 
on  physical  questions  than  on  actual  numerical  analysis. 
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9.  Inclusion  Theorems  for  Eigenvalues 

H.  Wielandt ' 

The  Problem 

Though  there  exists  a  large  number  of  inchision  theorems  for  the  eigenvalues  of  matrices,  little 
}ffort  seems  to  have  been  made  to  develop  what  we  might  call  an  inclusion  theory,  that  is  to  say  a 
'systematic  theory  giving  a  full  account  of  all  possible  inclusion  theorems  that  may  be  based  on  given 
issumptions.     The  following  remarks  are  intended  to  give  some  contributions  toward  such  a  theory. 

Of  com'se,  we  cannot  expect  to  succeed  in  giving  a  unified  theory  covering  all  kno^vn  inclusion 
theorems.  There  are  too  many  different  types  of  premises  involved.  We  therefore  shall  confine  our 
attention  to  one  definite  type  of  inclusion  theorems,  which  may  be  considered  to  be  the  most  natural 
one.  It  is  that  type  we  meet  in  principle  every  time  we  have  solved  an  eigenvalue  problem  Ax^'kc 
approximately. 

Suppose  we  have  obtained  some  vector  x  and  some  scalar  Xq  satisfying  Ax^XqX.  Then  the  question 
arises:  Wliat  do  we  know  about  the  true  eigenvalues  of  A?  We  may  expect  to  find  some  eigenvalue 
\  of  .4  to  lie  within  some  neighborhood  of  Xq,  the  extent  of  this  neighborhood  depending  on  the  degree 
of  accuracy  of  the  above  approximation.  This  degree  of  accuracy  should  be  measured  in  some  way  or 
another,  if  we  want  to  obtain  numerical  inclusion  theorems.  There  are  several  ways  of  doing  this. 
But  since  we  want  our  theory  to  be  as  general  as  possible,  we  prefer  to  avoid  a  decision  in  favor  of  one 
of  these  possibilities.     So  we  state  our  problem  in  the  following,  more  general  form: 

Let  us  be  given  two  vectors  x,y  of  n  complex  components.  (For  the  sake  of  simplicity  we  suppose 
khe  length  of  x  to  be  l:||a;|p=x*x=l,  and  y  to  be  linearly  independent  of  x).  Then  we  ask:  What  restric- 
\tions  are  imposed  upon  the  eigenvalues  of  an  n-rowed  matrix  A  by  postulating  Ax=y? 

The  first  answer  to  this  question  is  disappointing.  For  it  is  easy  to  prove  that  there  is  no  restriction 
at  all.  Even  if  the  given  vectors  x,y  are  proportional  with  an  extremely  high  degree  of  accm-acy,  we 
may  choose  n  arbitrary  numbers  Xi,  Xa,  .  .  .,  X„  and  still  find  a  matrix  A  possessing  these  values  as  eigen- 
values and  carrying  x  into  y.  So  if  we  want  to  get  any  information  about  the  eigenvalues  of  A,  we  have 
to  impose  on  A  some  additional  condition.  Here  arbitrariness  is  inevitable.  Let  us  take  as  starting 
point  that  class  of  matrices  whose  eigenvalue  theory  is  best  known. 

Heriiiitiaii  Matrices 

Let  ^  be  a  Hermitian  matrix.     In  this  case  there  are  several  well-known  inclusion  theorems  of  the 

type  considered.     For  instance,  if  we  let  the  real  number  x*y=n,  then  there  is  at  least  one  eigenvalue 
;of  A  to  the  left  of  n:  Xi<  m,  and  another  to  the  right:  X2>  m-     Further,  if  we  define  a  positive  number  r 
;by  r^=\\y\\^—fi^,  then  at  least  one  eigenvalue  satisfies  the  inequality   \\3  —  n\<r,  first  given  by  D.  N. 
iWeinstein  in  the  case  of  differential  equations. 
'         Now  let  us  attack  the  problem  of  determining  all  possible  inclusion  theorems  for  Hermitian  matrices 

First  of  all  we  have  to  make  the  problem  precise.     This  may  be  done  by  introducing  two  concepts,  that 

of  compatible  spectra  and  that  of  inclusion  sets. 

We  shall  denote  by  the  term  compatible  spectrum  the  set  of  the  eigenvalues  of  every  Hermitian 

matrix  A  that  carries  the  given  vector  x  into  the  given  vector  y.     So  Xi,  .  .  .,  X„  are  a  compatible  spectrum, 

if  there  exists  some  matrix  A  with  the  properties 

A=A*,    Ax=y,     |X/-yl|=n  (X-X,). 


University  of  Tiibingen,  Germany. 


75 


Using  this  definition,  we  may  say  that  each  of  the  three  theorems  quoted  determines  a  certain  subset 
of  the  X-axis,  which  contains  at  least  one  point  out  of  every  compatible  spectrum.  Such  a  subset  shall 
be  called  an  inclusion  set.  (Of  course  both  of  these  concepts  depend  on  the  vectors  a;  and  y.  But  since 
we  shall  keep  these  vectors  fixed  throughout  our  investigation,  we  shall  not  mention  this  dependence 
further.)     Now  we  are  able  to  make  our  problem  precise:  Given  x  and  y,  to  determine  all  inclusion  sets. 

It  is  perhaps  surprising  that  this  problem  may  be  settled  completely.  In  fact,  we  can  even  go  a 
step  further  and  determine  all  compatible  spectra.  The  solution  of  this  problem  takes  its  simplest  form 
in  geometrical  terms.  Let  C  denote  the  circle  with  center  m  and  radius  r  defined  above.  C  is  determined 
by  X  and  y.  If  we  project  the  X-axis  stereographically  upon  this  circle,  then  to  every  spectrum  X,,  .  .  .,  X„ 
corresponds  a  group  of  n  not  necessarily  distinct  points  of  C.     Now  the  following  theorem  holds. 

A  spectrum  is  compatible  ij  and  only  if  the  convex  closure  of  the  corresponding  point  set  of  C  contains 
the  center  n  oj  the  circle.  In  other  words,  incompatible  spectra  are  those  that  may  be  included  in  a  cir- 
cular segment  of  angle  less  than  tt.     From  this  we  can  easily  derive  all  inclusion  sets: 

A  given  subset  M  oj  C  is  an  inclusion  set  ij  and  only  ij  it  contains  a  semicircle.  So  the  semicircles 
are  the  minimal  inclusion  sets,  in  the  sense  that  we  cannot  omit  a  single  point  without  destroying  the 
inclusion  property. 

From  the  inclusion  sets  on  the  X-circle  C  we  obtain  the  inclusion  sets  on  the  X-axis  b}'  simply  pro- 
jecting back. 

Thus  all  possible  inclusion  theorems  of  the  type  considered  are  found  and  now  it  is  an  eas}'  task 
to  choose  for  every  purpose  and  every  supplementary  knowledge  available  the  most  appropriate 
inclusion  theorem. 

For  instance,  if  we  are  searching  for  the  best  upper  bound  ^  for  the  smallest  eigenvalue  and  /3  is  to 
depend  on  x  and  y  only,  choose  the  minimal  inclusion  set  for  which  the  upper  bound  is  as  low  as  possible. 
This  becomes  the  half-axis  X<j(i  hence  i8=M-  Or  if  we  are  interested  in  finding  an  approximate  eigen- 
value, for  which  we  can  guarantee  a  minimal  absolute  error,  then  we  have  to  take  the  center  of  the  in- 
clusion interval  of  smallest  length;  this  is  Weinstein's  interval;  so  the  approximate  eigenvalue  with 
smallest  absolute  error  is  n. 

The  Hermitian  case  being  settled,  we  look  for  generalizations. 

Normal  Matrices 

In  the  Hermitian  case  the  proofs  are  based  on  the  fact  that  Hermitian  matrices  are  characterized 
by  A=U~^DU,  D  real  diagonal,   U  unitary. 

Since  the  reality  condition  is  not  very  important  in  the  proof,  the  easiest  way  for  generalization 
is  to  drop  this  condition.  It  is  well  known  that  the  remaining  conditions  characterize  the  normal  mat- 
rices. If  we  generalize  the  concepts  of  compatibility  and  inclusion  sets  by  admitting  normal  matrices 
instead  of  Hermitian  matrices,  and  by  considering  complex  spectra  and  inclusion  sets,  then  the  proofs 
run  exactly  as  before,  and  we  arrive  at  the  following  result: 

Let  the  complex  number  n  and  the  positive  number  r  be  defined  by 

^=x*y,     r''=\\y\f-\^,\\ 

Let  the  complex  plane  be  projected  stereographically  upon  the  sphere  S  with  center  p,  and  radius  r.  Then 
a  given  sjiectrum  is  compatible  ij  and  only  ij  the  convex  closure  oj  the  corresponding  point  set  on  the  sphere 
contains  the  center  n  oj  the  sphere.  In  other  words,  exactly  those  spectra  are  incompatible  that  may  be 
included  in  a  spherical  segment  less  than  a  hemisphere.  Now  again,  it  is  easy  to  determine  all  inclusion 
sets:  A  given  subset  M  oj  S  is  an  inclusion  set  ij,  and  only  ij,  it  contains  a  hemisphere.  The  hemispheres 
are  the  minimal  inclusion  sets  yielding  the  optimal  inclusion  theorems.  So  the  normal  case  is  settled 
in  a  satisfactory  way,  and  we  may  turn  to  further  generalization. 

Norinalizable  Matrices 

Let  us  drop  the  condition  that  U  be  unitary.  We  obtain  the  class  of  all  matrices  that  are  conjugate 
to  diagonal  matrices.    They  ma}-  be  described  in  other  words  as  the  matrices  possessing  n  linearly  inde- 
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; pendent  eigenvectors  or  as  the  matrices  possessing  only  linear  elementary  divisors.  Let  me  refer  to 
I  these  matrices  as  the  normalizahle  matrices.  This  class  of  matrices  is  a  very  extensive  one,  although 
it  still  does  not  contain  all  of  them.  In  fact,  it  is  too  large  in  order  to  yield  inclusion  theorems  valid 
'throughout  this  class. 

Here  a  new  idea  enters.  We  may  hope  to  obtain  inclusion  theorems  again,  if  we  define  some  measure 
for  the  deviation  of  a  given  normalizahle  matrix  from  normality  and  then  restrict  the  class  of  matrices 
to  be  considered  by  limiting  the  deviation.  There  are  many  ways  of  measuring  deviation  from  normality, 
but  it  is  not  quite  easy  to  find  a  definition  suitable  to  eigenvalue  inclusion  theory.  We  proceed  as  follows. 
We  define  real  angles  between  any  two  vectors  a,h  of  the  complex  linear  space  in  which  the  eigenvectors 
lare  to  be  found  in  the  following  way: 

II       ^  7y  I 

jThen  we  consider  all  the  matrices  U  transforming  A  into  diagonal  form.  If  one  of  these  matrices  pre- 
serves all  right  angles,  then  it  is  a  scalar  multiple  of  a  unitary  matrix;  in  this  case  A  is  normal.  This 
,suggests  to  measure  the  deviation  of  A  from  normality  in  the  general  case  by  measuring  the  deformation 
of  right  angles,  which  is  necessary  for  transforming  A  into  diagonal  form.    To  carry  this  out,  we  define 

ffor  each  nonsingular  matrix  U &  deformation  angle  5i;=min  <^  (Ux,  Uy),  (  ^  {x,y)=-x  \  and  for  each  normal- 

izable  A  we  define  a  normality  angle  i'A=max  bu,  (?7~^ylC7=  diagonal).     So  we  have  0<j'a^o,  where 

the  equality  sign  holds  for  normal  matrices  only.  A  small  value  of  va  means  a  large  deviation  of  A  from 
normalit}'. 

Now  we  may  generalize  the  former  concept  of  compatibility  to  what  we  shall  call  ^-compatibility, 
where  v  denotes  some  fixed  angle  in  the  interval  0<^j'<x/2.  A  set  of  n  complex  numbers  Xi,  .  .  .,  X„ 
shall  be  called  a  v-compatible  spectrum,  if  there  exists  a  normalizahle  matrix  A  with  the  properties 

n 

Ax  =  y,     |X/— .41  =  n(X  — Xt),     va>v. 

k  =  l 

We  propose  to  determine  all  j'-compatible  spectra.  The  solution  of  this  problem  is  described  most  con- 
!veniently,  using  the  same  number  sphere  S  determined  by  x  and  y  that  we  introduced  in  the  normal 
case.  For  the  sake  of  convenience,  let  us  define  the  spherical  radius  p(Xi,  .  .  .,  Xn)  of  a  spectrum  to  be 
the  spherical  radius  of  the  smallest  segment  of  S  containing  all  the  X^,.  Then  we  may  state  the  following 
theorem. 

Main  theorem:     Xi,  .  .  .,  X^  are  a  v-compatible  spectrum  ij  and  only  ij  p(Xi,  .  .  .,  X„)>i'. 

In  other  words,  suppose  we  are  given  vectors  x,y  and  an}-  spectrum  Xi,  .  .  .,  X„  possessing  any  spheri- 
cal radius  p  upon  the  number  sphere  determined  by  x  and  y.  Then  there  exists  a  matrix  A  with  normality 
iingle  va=p,  which  possesses  the  spectrum  Xj,  .  .  .,  X„  and  carries  x  into  y\  but  there  is  no  such  matrix 
jwith  normality  angle  va^'p- 

j  The  v-compatible  spectra  being  known,  there  is  no  difficulty  in  determining  the  corresponding 
[.nclusion  sets.  The  minimal  inclusion  sets  are  the  spherical  segments  with  spherical  radius  ir—v;  every  other 
^Inclusion  set  contains  a  minimal  one.  So  all  possible  inclusion  theorems  arc  known,  and  it  would  be  easy 
':o  generalize  all  the  well-known  special  inclusion  theorems  holding  for  real  symmetric  matrices  to  the 
jiase  of  normalizahle  matrices. 

Generalizations 

After  the  normalizahle  case,  we  ought  to  consider  tlie  most  general  case,  imposing  on  A  no  other 
'(•estriction  than  a  certain  limitation  of  the  deviation  from  normality.  This  case  is  rather  more  difficult 
I'han  the  foregoing,  owing  to  the  fact  that  now  elementary  divisors  of  higher  degree  arc  allowed  to  occur. 
\i.s  this  investigation  has  not  yet  been  finished,  I  should  like  to  confine  myself  to  indicating  the  method 
employed.     It  is  essentially  a  geometrical  one. 

77 


The  main  theorem  stated  before  impUes  a  remarkable  geometrical  fact.  We  remember  that  the 
normality  angle  v  was  defined  to  be  an  angle  in  the  n-dimensional  space,  in  which  the  eigenvectors  are 
to  be  found.  In  the  theorem,  the  same  angle  appears  in  a  certain  three-dimensional  space,  namely,  the 
interior  of  a  certain  complex  number  sphere,  upon  which  the  eigenvalues  are  to  be  found.  To  put  it 
briefly,  one  and  the  same  angle  plays  a  fundamental  role  in  the  eigenvector  space  and  in  the  eigenvalue 
space.  This  surprising  result,  found  by  calculation,  caused  me  to  search  for  immediate  geometrical 
connections  between  the  eigenvector  space  and  the  eigenvalue  space.  Such  relations  exist  indeed.  The 
link  connecting  the  eigenvector  space  V  and  the  eigenvalue  space  N  is  the  tliree-dimensional  set  F  of 
those  points,  whose  rectangular  coordinates  may  be  expressed  by  ^=Re  x*Ax,  77=Im  x*Ax,  ^=x*A*Ax, 
where  x  runs  through  all  vectors  of  length  1  in  F.  F  is  a  convex  set  containing  on  its  boundary  the  n 
points  P,  with  coordinates  ^=Re  X;,,  r]  =  lm  K,  f=  |X„p  corresponding  to  the  spectrum  of  ^.  F  is  the 
convex  closure  of  these  n  points  if  and  only  if  A  is  normal.  F  lies  in  the  interior  of  the  paraboloid  P  with 
the  equation  f=^^+77^.  P  can  be  mapped  upon  the  number  sphere  S  by  an  appropriate  collineation 
In  this  way  a  mapping  of  the  eigenvector  space  V  into  the  eigenvalue  space  A^  is  defined.  The  geometrica  i 
relations  between  these  two  spaces  may  be  expressed  most  adequately  in  terms  of  hyperbolic  geometry 
At  the  present  time,  they  provide  the  most  powerful  tool  for  research  in  this  field. 

I  should  like  to  finish  by  indicating  two  further  generalizations  of  the  problem  we  dealt  with.  The 
first  consists  in  assuming  more  than  one  vector  and  its  image  to  be  given,  say  Xi,  .  .  .,  Xg  and  Axi,  .  .  ., 
Axs-  What  then  can  be  said  about  the  spectrum  of  ^4?  Here  even  the  case  of  Hermitian  matrices  is 
highly  complicated  and  up  to  the  present  time  not  perfectly  cleared  up.  However,  in  the  important 
special  case,  that  some  iterates  of  one  single  vector  are  given,  say  Xq,  Axq=Xi,  .  .  .,  Axs-i=Xg  {A=A*), 
a  satisfactory  survey  of  all  inclusion  sets  has  been  obtained.  For  example,  I  mention  that  for  every  given 
real  number  ^  the  shortest  inclusion  intervals  with  left  or  right  end  ^,  say  r)i<\<^^  and  ^<^\^V2,  can  be 
calculated  by  solving  the  following  algebraic  equation  of  degree  s  for  the  two  roots  771,  770  nearest  to  ^: 
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=  0. 


The  matrix  elements  are  defined  by  mk=Xi  *  Xj     (i-{-j=k). 

The  second  extension  I  want  to  mention  consists  in  considering  inclusion  theorems  not  only  for  eigen- 
values, but  for  eigenvectors  too.  Because  this  problem  goes  beyond  the  limits  of  the  present  talk,  I  shah 
confine  myself  to  mention  the  rather  surprising  fact  that  it  can  be  solved  in  a  perfectly  satisfactory  waj 
for  those  classes  of  matrices  for  which  a  full  eigenvalue  inclusion  theory  has  been  developed. 

There  is  no  difficulty  m  extending  the  theory  to  the  case  of  operators  in  Hilbert  space. 
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10.  On  General  Computation  Methods  For  Eigenvalues 

and  Eigenfunctions' 

Gaetano  Fichera^ 

Methods  for  computation  of  eigenvalues  of  symmetric  or  Hermitian  quadratic  operators  in  Hilbert 
spaces  have  now  been  so  fully  developed,  that  eigenvalue  problems  arising  in  the  applications,  related  to 
self-adjoint  linear  equations,  can  be  considered — at  least  from  a  theoretical  standpoint — as  solved. 
However,  in  some  cases  computation  of  eigenvalues  of  nonsymmetric  operators  is  needed. 

"We  want  to  sketch  here  two  procedures  that  have  been  applied  at  the  Italian  Institute  for  the 
Applications  of  Calculus  in  connection  with  quite  general  linear  eigenvalue  problems.  The  first  of  these 
can  also  successful!}"  be  applied  even  in  self-adjoint  problems,  in  which  classical  procedures  are  impractical 
because  of  the  difficulty  of  getting  complete  sets  of  fmictions  satisfying  certain  boundary  conditions. 

General  Least  Squares  Method 

Let  £r  be  a  separable  Hilbert  space,  linear  with  respect  to  the  complex  field.  If  u  is  an  element  of 
H,  u  denotes  the  complex  conjugate  of  u.  Let  us  denote  m  the  usual  way  the  scalar  product  of  two 
elements  u  and  v  of  H  by  ('/.r).    It  is  well  known  that  the  scalar  product  has  the  followmg  properties: 

(a)  (u,v)  =  iv,u), 

(b)  {u,ai\-\- fiv2)  =  a(^u,Vi)  +  ^(u,V2) ,  {a  and  /3  complex  numbers), 

(c)  {u,v)  =  (u^), 

(d)  |Iw||^=(w,it)  is  real  and  positive  when  u  does  not  coincide  with  the  origin  co  of  H. 
hetL[u]  and  M[u]  be  two  operators  that  map  H  or  part  of  it,  respectively,  m  two  linear  subspaces  of  H. 
Then 

L[aUi-T  ^u^]=aL[ui]-r  PL[U2\ 

In  addition  we  suppose 

lim||iM||  =  0,         lim||M[w]||  =  0. 

!i«n-*o  iiuii-o 

We  start  with  the  following  eigenvalue  problem 

L[u]  +  \M[u]  =  u:         \\u\\  =  l.  (1) 

Let  us  consider  the  followuig  real-valued  functional 

P(u,\)  =  \\L[u]  +  \M[uW. 

It  is  evident  that  P{u,X)  vanishes  on  the  unit-sphere  ||w|i  =  l  when  and  only  when  an  eigenvalue  of  the 
problem  (1)  exists.    Let  {vk)  then  be  a  complete  system  of  points  of  H. 

Generally  the  set  of  the  eigenvalues  for  problem  (1)  is  a  set  of  points  of  the  complex  plane  without 
finite  limit  point.  We  shall  assimie  this  hypothesis  for  problem  (1)  and  denote  by  Xj,  X2,  .  .  .,  X„,  .  .  ., 
the  sequence  of  the  eigenvalues  ordered  in  such  way  that  |X„|<|X„+i|  and  if  lX„|^|X„+i|,  then 
princ.  arg  X„<^princ.  arg  X„+i. 

Let  us  set 

i^<'"  =  i:c,v»,  (2) 

ik  =  l 

l,n 

PrXCuC2,-    •    .,C„,X)  =  P(w„,X)  =  S^A*CACt, 

h.k 


'  The  methods  discussed  in  this  paper  were  given  by  Prof.  M.  Picone  and  applied  at  the  Italian  Institute  for  the  -Applications  of  Calculus. 
•'  University  of  Trieste  and  Italian  Institute  for  the  Applications  of  Calculus. 
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AnK=^{L[Vn],L[v,])  +  \{M[Vj],L[Vk\)^-\{M[v^],L[Vn])  +  \HM[Vn],M[v,]), 


Q„(c,,c„...,c„)  =  |k^"'IP=g|c.|^ 


F„  and  Qn  are  two  Hermitian  quadratic  forms  in  the  variables  Ci,  C2,  .  •  •,  Cn- 

We  want  to  get  the  minimum  of  Pn  under  the  condition  0„=1.     We  are  thus  ,led  'according  to  the 
theory  of  Hermitian  forms  to  the  following  Hnear  algebraic  system  in  the  unknowns  Ci,  C2,  .  .  .,  c„: 


k=l 

Let  Z)(X,X,m)  be  the  determinant  of  system  (3).     We  get  the  equation 


(3) 


(4) 


For  every  fixed  X,  equation  (4)  has,  as  is  well  known,  n  real  roots. 

If  (3)  and  (4)  are  satisfied,  then  P„— ^^^=0.  From  the  condition  0„=1,  it  follows  Pn=fJi-  Let  X|"', 
^2"',  •  •  •,  ^t"'  be  the  minimizing  points  for  the  functions  ^l(\)  implicit!}^  defined  b}^  (4).  We  suppose 
that  these  values  are  ordered  in  the  same  way  as  the  sequence  Xi,  X2,  .  .  .,  X„,  .  .  ..  The  values  so 
obtained  are  taken  as  nth  approximation  of  the  first  j/„  eigenvalues  of  problem  (1). 

Of  coiu'se,  a  theoretical  justification  of  this  procedure  is  needed  and  the  method  at  the  actual  state 
is  quite  empirical.  However,  the  numerical  experiences  performed  in  the  Italian  Institute  in  man}'  par- 
ticular problems  gave  always  very  satisfactory  results  not  only  for  the  determination  of  the  eigenvalues 
but  also  for  the  computation  of  the  corresponding  eigenfunctions. 

We  want  to  report  here  the  numerical  results  obtained  in  a  problem,  in  which  other  procedures  are 
applicable  too,  in  order  to  compare  the  results  of  all  of  them. 

The  problem,  which  is  connected  with  an  engineering  problem,  is  the  following:  ^ 


[{l-9xy^^^-\il-ex)u  =  0,  ru'dx=l,         u{0)=u{l)=u"{0)=u"{l)  =  0. 


(5) 


I 


^  is  a  numerical  nonnegative  constant  less  than  1.    In  the  case  ^=0,  the  eigenvalues  X„  and  the  corre- 
sponding eigenf unction  Un(x)  are  known: 

X„=7iV,         u n{x)  =  ■^J 2  sin  nirx. 

The  first  approximation  gives  the  following  approximation  for  the  first  eigenvalue 

3024 


Xi^ 


31 


1-0- 


42 


In  the  case  6=0,  we  get  Xji^ 97.548,  while  the  exact  value  is  Xi  =  7r*= 97.4091.  The  following  table 
shows  the  numerical  results  obtained  in  the  case  6=0  for  the  first  three  eigenvalues,  compared  with 
their  exact  values. 


First 
approximation 

Second 
approximation 

Third 
approximation 

Exact  value 

X3 

97.  548 

97.  548 
1584.  004 

97.  4091 
1584.  004 
7890.  136 

97.  4091 
1558.  545 
7890.  13 

'  T.  Viola,  Calcolo  approssimato  di  antovalori,  Rend,  di  Mat.  e  delle  sue  appl.    S.  V.  II,  (1941). 
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The  following  table  compares  the  values  obtained  by  the  method  in  second  and  third  approximation 
for  the  first  ei2:enfunction  with  the  exact  values: 


^u.(x) 

sin  ttx 

Second 
approximation 

Third 
approximation 

0 
0.  1 
0.2 
0.3 
0.4 
0.5 

0.6 
0.7 
0.8 
0.9 

1 

0 
0.  308  190 
0.  583  080 
0.  798  279 
0.  934  938 
0.  981  748 

0.  934  938 
0.  798  279 
0.  583  080 
0.  308  190 
0 

0 
0.  309  003 
0.  587  644 
0.  808  599 
0.  950  353 
0.  999  160 

0.  950  353 
0.  808  599 
0.  587  644 
0.  309  003 
0 

0 
0.309  017 
0.  587  785 
0.  809  017 
0.  951  057 

1 

0.  951  057 
0.  809  017 
0.  587  785 
0.  309  017 
0 

For  9=0.5  we  have  the  following  table  of  the  values  obtained  in  four  approximations  for  the  first  three 
eigenvalues: 


First 
approximation 

Second 
approximation 

Third 
approximation 

Fourth 
approximation 

Xi 

X2 
X3 

48.  194 

48.  720 
852.  95 

50.  890 
838. 

50.  8129 
836.  9412 
4256.  878 

Tricomi  *  gives  for  Xi  the  lower  bound  47.9215  and  the  upper  bound  57,1553.  We  shall  compare  the 
obtained  numerical  results  with  those  obtained  by  using  the  method  of  the  next  section. 

I  Method  Founded  on  Cauchy-Lipschitz  Integration  Procedure 

I  The  present  method  is  in  connection  with  eigenvalue  problems  for  normal  systems  of  ordinary 

linear  differential  equations.     We  can  suppose  the  system  to  be  of  the  first  order,  and  we  write  it  in 
i  the  following  vectorial  form 

^=H{t,\).x(t).  (6) 

i(t)  is  a  vector  function  of  the  real  variable  t,  in  an  interval  (a,^)  having  p  real  or  complex  components 
Xft(0  (A.=  l,2,.  .  .,  p).  H{t,\)  is  a  square  matrix  of  order  p.  Its  elements  Hhk(t,  X)  depend  con- 
tinuously upon  the  real  variable  t  and  the  complex  varia])le  X.    H{t,  \)*x{t)  denotes  the  2>-vector  having 

p 
as  hth.  component  X)  TIhk(t,X)Xk(t). 

k  =  l 

Let  Li[x],  L2U],  .  .  .,  ip(a:)  be />  functional  depending  linearly  upon  x,  that  is  to  say, 

Li\ax-{-by]=aLi[x\  +  hL  i[y] 
for  every  pair  of  complex  constants  a  and  h. 

•  F.  Tricomi,  Siille  vibrazionai  trasversali  di  aste,  specialmonte  di  biollc  di  sc/.ionc  variabile,  Kiwrclic  di  Ingeeinoria  [2]  IV,  47  (Mar/o-Aprile  1930). 
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We  consider  the  following  eigenvalue  problem 

dx 


dt 
L 


Hit,  \)*x{t) 
,[x]  =  0     {i=l,2,...,p),    jjx{t)\'dt=l. 


(7) 


Let  x'-''\t,X)  be  the  integral  of  (6)  determined  by  the  initial  condition 


(8) 


The  general  integral  of  (6)  is 


x{t,\)==^  c,x''\t,\). 


fc=l 


A  necessary  and  sufficient  condition  for  the  existence  of  eigenvalues  of  problem  (7)  is  the  existence  of 
roots  of  the  equation 

A(X)  =  ||i,[a;('^>(^,X)]||  =  0. 

Let  2^*'''^(^,X)  be  the  vector  obtained  by  applying  the  Cauchy-Lipschitz  approximation  method  in  the 
nth  step  with  the  initial  condition  (8) . 
Let  us  set 

x^''Kt,\)=J^ct'x^''"^{t,\),  and  AnCK)=\\Li[x^''-"^{t,\)]\\  =  0. 

k  =  l 

If  Li[x]{i=l,2,  .  .  .,p)  is  supposed  to  be  a  continuous  functional  of  the  first  order  of  x,  that  is  to  say 
if  lim|i[a:]|  =  0,  if  (|i;|  +  |x'|)-^0,  and  if  Hhkif,^)  has  continuous  first  deriv^atives  with  respect  to  t  and  is 
an  holomorphic  function  of  X  in  the  whole  complex  plane,  then  it  has  been  shown  ^  that  every  root  of 
A(X)  is  approximated  by  a  root  of  A„(X). 

The  method  described  was  applied  to  the  problem  (5).  Putting  n  =  10,  that  is  to  say,  by  applying 
the  Cauchj^-Lipschitz  procedure  when  the  interval  (a:,jS)  is  divided  in  10  parts,  it  furnished  the  following 
values  for  the  first  three  eigenvalues  in  the  case  0=0.5:  Xi=50.9,  X2=837.4,  X3=4212.2.  The  follow- 
ing table  compares  the  approximate  value  for  the  first  eigenfunction  of  the  problem  (5),  obtained  by 
least  squares  method  and  Cauchy-Lipschitz  procedure  in  the  case  6=0.5: 


Ui 

ix) 

X 

Least  squares 

Cauchy-Lipscbitz 

0 

0 

0 

0.  ] 

0.  099  17 

0.  099   17 

0.2 

0.  192  94 

0.  192  97 

0.3 

0.  274  86 

0.  274  91 

0.4 

0.  337  70 

0.  337  67 

0.5 

0.  373  86 

0.  373  58 

0.  6 

0.  376  16 

0.  375  47 

0.7 

0.  338  95 

0.  337  87 

0.8 

0.  259  92 

0.  258  69 

0.  9 

0.  142  62 

0.  141  63 

1 

0 

0 

•  T.  Viola,  Dimostrazione  della  convergenza  di  un  procedimento  di  M.  Picone  per  il  calcolo  degli  autovalori,  Rend  Accademia  Kaz  Lincci  XXIX  (1939). 
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11.  Variational  Methods  for  the  Approximation  and 
Exact  Computation  of  Eigenvahies  ^ 

Alexander  Weinstein " 
11.1.    Introduction 

The  variational  methods  for  the  approximation  of  eigenvakies  have  ahvays  been  intimately  connected 
with  the  development  of  the  theory  of  membranes  and  plates.  In  fact,  the  variational  definition  of 
eigenvalues  so  often  used  today  goes  back  to  a  classical  paper  of  H.  Weber  on  the  vibration  of  membranes 
which  was  published  as  the  first  paper  in  the  first  volume  of  the  Mathematische  Annalen.  The  methods 
of  Rayleigh-Ritz  were  first  developed  for  plates.  In  the  decade  1910-20  the  minimax  theory  and  the 
asymptotic  laws  for  the  distribution  of  eigenvalues  were  first  developed  for  membranes  and  plates  [1,  2]. 
More  recently,  the  same  problems  led  to  methods  for  the  variational  determination  of  upper  and  lower 
bounds  for  eigenvalues  which  resulted  in  a  unified  theory  of  eigenvalues  of  plates  and  membranes  [3,  41. 
I  Although  it  would  therefore  be  justified  to  present  the  theory  of  the  approximation  of  eigenvalues 
'  in  terms  of  differential  operators,  we  shall  use  mostly  the  theory  of  integral  operators  which  are  the-  in- 
verse of  the  differential  operators.  In  other  words,  we  shall  deal  with  completely  continuous  operators  in 
a  Hilbert  space.  In  the  present  paper  we  \vill  consider  only  methods  that  lead  to  arbitrarily  precise  ap- 
proximations of  eigenvalues,  and  will  not  be  concerned  with  rough  inequalities. 

I  11.2.  Operators  in  a  Hilbert  Space  and  Its  Subspaces 

Let  H  be  a  real  Hilbert  space  and  let  L  be  a  positive  definite  completely  continuous  symmetric 
operator.     We  shall  use  the  standard  notations  of  the  theory  of  Hilbert  space.     The  eigenvalue  equation 

j  Lu  =  \u  (1) 

possesses  an  uifinite  number  of  positive  eigenvalues 

'I  xr>xr>  •  •  •;    lii"  X^°'=0  (2) 

T»— »co 

with  the  corresponding  normalized  eigenvectors 

<>,     iif ,     .  .  ..  (3) 

It  is  well  known  that  these  eigenvalues  can  be  obtained  by  a  chain  of  maximum  problems.     For  instance, 

X <°'  =max  ^^^  for  u  inH.  (4) 

{u,u) 

The  second  eigenvalue  X^^  is  the  maximum  of  the  same  expression  for  v  orthogonal  to  yl°\  and  so  forth. 
Let  Q  be  a  closed  linear  subspace  oi  H,  and 

'I  P  =  HeQ  (5) 

Consider  the  corresponding  chain  of  maximum  problems  for  the  exi)ression  (r,Zr)/(r,r)  where  r  is  restricted 
to  Q.     The  maximizing  vectors  again  satisfy  an  eigenvalue  equation  of  the  type 

Lv-\v=p,  (6) 


I  This  work  was  performed  under  the  sponsorship  of  the  Office  of  Naval  Hesearch. 

'  Institute  for  Fluid  Dynamics  and  Applied  NTathematics,  Vniversity  of  Maryland,  College  Park,  Maryland. 
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where  p  is  some  element  of  P.  In  order  to  put  (6)  in  the  same  form  as  (1),  we  take  the  projections  of 
both  sides  into  the  space  Q.  The  projection  of  Lv  is  itself  a  completely  continuous  positive  operator 
in  Q,  which  we  call  L'v.     In  this  wa}^  we  obtain  in  place  of  (6)  the  eigenvalue  equation 

X'r-Xr=0  (7)     , 

\t 

x;>x;>  .  .  .;   x:-^o  (8) 


which  admits  the  eigenvalues 

with  the  corresponding  eigenvectors 


Ui,       U2, 


(9) 


The  minimax  theory  gives  the  fundamental  inequalities 

xr>x:;     n  =  l,     2,     .  .  ..  (10) 

Let  us  note  that  these  well-known  inequalities  include  the  Rayleigh-Ritz  method.  In  fact,  if  Q  is  a 
space  of  a  finite  number  of  dimensions,  the  operator  L'  is  a  matrLx,  so  that  the  eigenvalues  X^°^  of  Z'  can  be 
considered  for  the  purpose  of  the  present  paper  as  known.  In  this  way,  low^er  bounds  for  the  X^°'  are 
given  by  (10).  (It  should  be  noted  that  the  eigenvalues  of  a  differential  operator  are  the  inverses  of 
the  eigenvalues  of  the  corresponding  L,  so  that  the  signs  of  all  inequalities  must  be  reversed.) 

Generally  speaking,  the  foUowdng  alternatives  may  be  encountered.  Either  the  eigenvalues  and 
eigenvectors  of  L  or  those  of  L'  are  knowTi.  In  the  first  case,  when  the  X^°'  are  known,  (10)  gives  upper 
bounds  for  the  eigenvalues  X^  of  L' .  In  the  second  case,  (10)  gives  lower  bounds  for  the  eigenvalues 
X,l°'  of  L.  Let  us  give  an  example  for  the  first  case.  Let  G  be  the  Green's  function  for  a  square  satisfying 
the  equation  AA6'=0  and  the  boundar}-  conditions  G=^G=0,  and  define  L  as  the  integral  operator  whose 
kernel  is  G.  Then  the  X^"^  are  explicitly  kno\\ai,  since  the  problem  is  then  essentially  that  of  a  vibrating 
membrane.  For  L'  wo,  take  as  the  kernel  the  Green's  function  satisfying  the  same  equation  AA6''=0 
and  the  boundary  conditions  G'  =  ()G'/dn  =  0.  This  last  problem  is  the  problem  of  the  vibrating  clamped 
square  plate.     The  subspace  P  tm-ns  out  to  be  the  space  of  harmonic  functions  of  integrable  square. 

For  the  second  case,  an  example  where  X^  is  knownti  is  given  by  the  problem  of  the  vibrations  of  a 
circular  clamped  plate.  The  operator  L  in  this  case  refers  to  the  vibration  of  a  circumscribed  square 
plate.  In  this  case,  the  subspace  P  is  the  space  of  functions  harmonic  within  the  circle  and  arbitrary 
in  the  remainder  of  the  square. 

11.3.  The  Intermediate  Problems 

We  now  introduce  the  concept  of  intermediate  problems,  which  will  lead  to  an  essential  improve- 
ment of  the  bounds  given  by  the  inequality  (10).  We  shaU  restrict  ourselves  to  the  consideration  of' 
upper  bounds.  This  will  be  done  not  only  to  avoid  repetition,  but  also  because  some  of  the  new  results 
given  in  this  paper  hav^e  only  been  derived  for  upper  bounds.  In  other  words,  we  consider  the  first 
case  of  the  preceding  section,  where  the  eigenvalues  X'°'  and  the  eigenfunctions  m'°'  of  L  are  kno\Mi. 
Let  (pi,  ^2,  .  .  •  )  be  a  basis  for  the  subspace  P,  that  is,  a  complete  set  of  linearly  independent  (but 
not  necessarily  orthogonal)  elements  of  P.  There  is  an  infinite  choice  of  such  bases,  but  for  the  time 
being  we  will  not  specif}*  any  particular  one. 

The  intermediate  problem  of  index  m  is  defined  in  the  following  way.     Introduce  the  space 

Hm=He{vi,  ■  ■  ■  ,vm].  (11): 

Consider  the  chain  of  maximum  problems  for  the  expression 

{v,Lv)l(v,v)  (12) 

with  V  restricted  to  lie  in  H^.  This  leads  to  an  eigenvalue  problem  of  the  form  (6),  where  p  must  in 
the  present  case  be  a  vector  in  the  finite  space  {^^i,  .  .  .  ,  Pm] ■     In  this  way,  we  get  the  equation 

iw«-X'f'U^r'=C'Vi+   .  •   .   +a':--p^,  (13) 
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'  where  the  constants  a^-'  are  unknown.     A^rain   taking   the   projection  into  Hm  ansd  calling-  i^'"^  the 
projection  of  L  into  Hn,  this  can  be  written,  similar  to  (7),  as 

Z<'"^u-Xw=0.  (14) 

This  equation  yields  the  eigenvectors  w'"'  and  the  eigenvalues 

I  \'t>\'f>    .   .   .;     limX<f=0.  (15) 

! 

'i  Since   Hm   contains  the  subspace   Q   and  is  contained  in  Hn-i  and  in  H,  we  have  by  the  minimax 
theory,  as  in  (10), 

I  X<°'>X'i'>X<;'>    .   .   .    >X'„;     n  =  \,2,   ....  (16) 

It  has  been  shown  that  for  each  fixed  n 

limX^:'=X:.  (17) 

!  It  would  perhaps  have  been  more  approjjriate  to  use  the  notation  Ho  for  H  and  H^  for  Q  and  to  ^^Tite 
:  in  place  of  L  and  L'  the  symbols  i"''  and  i^°°\  but  this  would  be  too  cumbersome. 

The  intermediate  problem  was  introduced  by  Weinstehi  [3].     The  question  of  convergence  was 

investigated  jointly  b}'  Aronszajn  and  Weinstein  [4]. 

The  fundamental  fact  about  the  intermediate  problems  is  the  following:  The  eigenvalues  and 
leigenf unctions  of  the  operator  Z^"*^  can  be  explicitly  computed  in  terms  of  the  corresponding  quantities 
'of  Z,  which  are  assumed  to  be  known.     The  problem  for  L  could  therefore  be  called  the  base  problem. 

The  problem  for  Z'(=i^"^),  cannot  be  explicith-  solved,  but  appears  as  a  limiting  case  of  explicitly 

solvable  intermediate  problems. 

The  discussion  and  solution  of  the  intermediate  problems  is  obviously  a  major  part  of  the  theory. 

It  was  given  for  a  special  sequence  (i?i,2>2i  •  •  •)  hy  Weinstein  [3]  in  the  case  of  the  vibrating  plate,  and 
Hater  substantially  improved  bj'  Aronszajn  [5,  6]  for  general  operators  in  Hilbert  space.     Aronszajn 

rightly  emphasized  the  fact  that  the  solution  of  the  intermediate  problems  can  be  given  by  the  use  of 

an  arbitrary  basis  (pi,P2,  .  .  .)•  ^e  shall  refrain  from  the  formulation  of  these  general  results,  inas- 
imuch  as  adequate  presentation  of  this  part  of  the  theory  has  been  given  in  recent  papers  mentioned 

above. 

The  method  of  upper  bounds  was  originally  introduced  by  Weinstein  as  the  counterpart  of  the 

Rayleigh-Ritz  method  for  lower  bounds.     Later,  Aronszajn  introduced  as  its  actual  counterpart  a 

generalized  Rayleigh-Ritz  method  in  which  the  space  Q  defined  in  section  11.2  may  have  an  infinite 
'number  of  dimensions.  He  showed  that  the  generalized  Rayleigh-Ritz  method  can  be  discussed  in 
hssentially  the  same  fashion  as  the  method  of  upper  bounds  [5,  6,  7].  'ilie  basic  idea  underlying  all 
jthese  new  developments  is  the  consideration  of  spaces  that  differ  only  by  a  finite-dimensional  space. 
1  The  main  purpose  of  the  following  paragraphs  will  be  to  bring  some  new  and  ])artly  unpublished 
jresiilts  going  beyond  the  question  of  solution  of  the  intermediate  problems  and  the  convergence  of  the 
iapproximations.     These  results  are  obtained  by  a  special  selection  of  the  l)asis  {jpu  P2  .   .  .  )■ 

11.4.  Weinberger's  Estimate  of  the  Error  in  the  Method  for  Upper  Bounds 

As  long  as  the  Rayleigh-Ritz  method  was  the  only  known  variational  method,  the  error  in  the 
lower  bounds  was  not  known.  By  combining  the  methods  for  lower  bounds  (either  Rayleigh-Ritz  or 
igeneralized  Rayleigh-Ritz  as  introduced  by  Aronszajn)  with  the  method  for  up])(>r  bounds,  we  can  get 
in  numerical  ai)plications  an  ai)])roximation  to  the  eigenvalues  with  a  known  error  which  can  be  made 
•arbitrarily  small.  However,  the  nunilxT  of  ste])s  recjuired  to  obtain  a  i)resciibe(l  i)recision  is  not  known 
a  priori,  and  is  left  to  chance. 

The  situation  would  naturally  be  (piite  different  if  an  estimate  of  tiie  error  were  known  for  at  least 
one  of  the  variational  methods.     Several  atteni])ts  were  math'  for  the  estimation  of  the  error  in  the 
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classical  Raj'leigh-Ritz  method  but,  as  far  as  is  knowm  to  the  author,  no  such  useful  error  estimate 
exists.  The  same  is  true  for  the  generalized  Rayleigh-Ritz  method.  However,  quite  recently  H.  F. 
Weinberger  [8]  has  given  an  error  estimate  for  the  method  of  upper  bounds  which  is  based  on  a  special 
selection  of  the  basis  (puP2,  ■  ■  .  ).  This  estimate  renders,  in  principle,  the  method  for  upper  bounds 
independent  of  the  methods  for  lower  bounds,  and  immediately  gives  the  number  of  steps  required  for 
a  giv^en  precision. 

Weinberger's  result  can  be  formulated  as  follows  in  the  notation  of  the  previous  paragraphs. 

Let  pn  be  the  projection  into  the  space  P  of  the  eigenvector  u'^\  Then  the  following  inequalities  hold 
-for  the  intermediate  problem  of  index  m. 

^'n'  >K    >>^'n'  -y°2+l-  (18) 

This  is  a  uniform  estimation  of  the  error  in  the  upper  bound  for  the  eigenvalues  X'„,  which  is  obtained 
by  the  use  of  the  intermediate  problem  of  index  m.  The  error  estimate,  which  is  equal  to  the  known 
eigenvalue  X'^'+i  oi  L,  goes  to  zero  for  increasing  w,  uniformly  in  n.  Of  coui'se,  for  each  m,  the  lower 
bound  for  X^  given  by  (18)  will  be  negative,  and  hence  trivial,  for  sufficiently  large  n.  But,  on  the 
other  hand,  for  each  eigenvalue  X'„  the  index  m  of  the  intermediate  problem  can  be  chosen  so  large 
that  the  lower  bound  in  (18)  is  positive. 

To  prove  Weinberger's  estimate,  we  again  consider  as  in  paragraph  3,  the  space 

Hm  =  He{Pl,     .     .     .,Pm]  (19) 

;  of  the  theorem.     Furthermore,  we  define  the  space  P^  by  the 

Pm  =  PQ{pi,     .     .     .,Pm].  (20) 


with  the  pn  defined  as  in  the  statement  of  the  theorem.     Furthermore,  we  define  the  space  P^  by  the 
equation 


Then  it  is  obvious  that 

Pm^HmOQ.  (21) 

Let  us  call  m'^'  the  eigenvalues  of  the  projection  of  L  into  P,„.     Then,  by  a  special  case  of  the  funda- 
mental inequality  due  to  Aronszajn  [5,  p.  476,  corollary  /'], 

X'n^<X:-M^f.  (22) 

For  an  arbitrary  choice  of  the  basis  {p,,p2,  .  .  .),  we  have  two  unkno\\iis  in  this  inequality,  namely, 
X„^and  ii'i.     But  for  Weinberger's  choice  of  the  p^,  the  space  P^  is  orthogonal  to  the  eigenvectors 


^T,  •  .  .,  u 


For  we  have,  for  any  vector  p  in  Pn, 


(P,U'n)  =  (P,Pn)=0      71=1,    .    .    .,    m.  (23) 

Therefore,  the  classical  variational  definition  of  eigenvalues,  applied  to  L,  yields  the  inequality 

^^^<KU     for     peP^,  (24) 

and  hence 

,,(m)  ^  \     (0)  /i-,-\ 

M     1     2i  f^m  +  l-  (20) 

This,  together  with  (22)  and  (16),  fields  the  inequahty  (18). 

The  problem  of  the  determination  of  error  estimates  is  therefore  reduced  to  the  determination  of 
the  projections  p„  of  the  eigenvectors  u^^\  which  will  be  discussed  elsewhere. 

11.5.  The  Optimum  Problem 

Since  the  intermediate  problems  depend  upon  the  choice  of  the  basis  (pi,p2,  .  .  .),  the  following 
question  arises  naturaUy.  This  question  will  be  formulated  here  onlv  in  the  simplest  case  of  the  first 
intermediate  problem.     What  is  the  lowest  value  of  the  upper  bound  X'i^  for  K  that  can  be  obtained  by  a 
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suitable  choice  oj p^"!  In  this  discussion  n  is  a  fixed  index.  Before  answering  this  question,  let  us  observe 
that  the  following  inequality  is  known  as  the  residt  of  a  separation  theorem  for  eigenvalues  [5,  p.  476, 
theorem  Bj: 

X'i'>Xn'+i-  (26) 

Also,  since  Xi^^  is  an  upper  bound  for  X^ ,  it  must  satisfy  the  inequality 

^'n'>K-  (27) 

As  an  answer  to  the  question,  H.  F.  Weinberger  has  shown  that  the  vector  px  can  be  so  chosen  that 
the  weaker  of  the  two  inequalities  (26)  and  (27)  becomes  an  equality.     In  other  words,  either 

X'i'=V°;i,  (28) 

or 

XT=X^  (29) 

The  proof,  which  involves  the  discussion  of  several  cases,  \nW  be  published  elsewhere  [91. 

The  solution  of  similar  optimum  problems  is  of  importance  to  the  computer  who  has  no  elaborate 
equipment  at  his  disposal. 

11.6.  An  Extension  of  the  Classical  Sturm-Liouville  Theory 

As  was  noted  in  section  11.3,  the  eigenvalues  and  eigenvectors  of  an  intermediate  problem  can  be 
exactly  computed  in  terms  of  those  of  the  base  problem  and  vice  versa.  The  theory  of  intermediate 
problems  was  originally  introduced  in  view  of  applications  to  partial  differential  equations.  In  this 
field,  they  play  an  auxiliary  role,  linking  by  an  infinite  chain  the  base  problem  to  the  problem  which  is 
to  be  solved.  However,  in  the  theory  of  ordinary  differential  equations,  the  intermediate  problems  have 
an  independent  significance. 

Some  important  chapters  of  the  classical  Sturm-Liouville  theory  deal  with  the  relations  between  the 
eigenvalues  and  eigenfunctions  of  two  problems  having  the  same  ordinary  differential  equation  but  dif- 
ferent boundary  conditions.  It  was  recently  pointed  out  by  Weinstein  [10,  11]  that,  for  a  large  class  of 
problems,  the  two  problems  in  question  can  be  interpreted  as  a  base  problem  and  an  intermediate  prob- 
lem which  terminates  a  finite  chain.  V>\  the  general  theory,  the  eigenvalues  and  eigenfunctions  of  either 
of  the  two  problems  can  be  explicitly  computed  in  terms  of  the  other.  This  result  yields  a  new  proof 
of  the  classical  separation  theorems  that  correspond  to  the  intermediate  problems  of  index  one,  but  goes 

\  beyond  that  b}'  replacing  classical  inequalities  by  precise  equations.     It  also  yields  separation  theorems 
of  higher  order  which,  to  our  knowledge,  were  not  discussed  in  the  classical  Sturm-Ijiouville  theory. 

Recently,  H.  F.  Weinberger  [12]  extended  the  explicit  solution  of  one  difi"erential  problem  in  terms 
of  another  to  all  pairs  of  self-adjoint  problems  having  the  same  ordinary  differential  operator  but  dif- 
ferent boundary  conditions.  Naturally,  in  some  of  these  cases,  the  separation  theorems  are  no  longer 
valid.  In  this  generality,  the  problems  cannot  always  be  interpreted  as  eigenvalue  problems  belonging 
to  a  space  and  its  subspace.  However,  it  can  be  showTi  that  they  correspond  to  two  spaces  which  are 
subspaces  of  finite  index  of  their  union.     Besides  its  application  to  Sturm-Liouville  theory,  these  new 

:  developments  constitute  a  remarkable  extension  of  the  thcor\"  of  intermediate  problems. 
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12.  Deteritiinatioii  of  Eigenvalues  and  Eigenvectors 

ot  Matrices 

Magnus  R.  Hestenes' 

12.1.  Introduction 

The  problem  at  hand  is  that  of  finding  a  number  X  and  an  ?i-dimensional  vector  y^O  such  that 
Ay=\y,  where  A  is  an  rt-dimensional  square  matrix.  The  vector  y  is  called  an  eigenvector  of  ^1  and  the 
number  X  is  called  an  eigenvalue  of  A.  Unless  otherwise  expressly  stated,  we  shall  be  concerned  with 
a  symmetric  matrix  A  over  the  field  of  reals.  The  extension  of  these  results  to  Hermitian  matrices 
over  the  complex  field  is  immediate. 

In  the  present  paper  we  describe  certain  methods  that  have  been  studied  at  the  Institute 
for  Numerical  Analysis,  National  Bureau  of  Standards.  Our  experiments  indicate  that  these  methods 
can  be  used  effectively  in  computations  for  symmetric  or  Hermitian  matrices.  The  Institute  is  sponsor- 
ing a  program  of  study  of  methods  of  finding  eigenvalues  and  eigenvectors  of  an  arbitrary  matrix. 

12.2.  The  Power  Method 

One  of  the  best-known  methods  of  finding  eigenvalues  is  the  power  method.  In  this  method  a 
sequence  of  vectors  {xi]  is  constructed  by  the  formula 

Xi+i=aiAxi         (i=0,l,2,  .  .  .). 

The  initial  vector  Xq  is  arbitrary,  and  the  numbers  at  are  scale  factors.  In  this  method  one  does  not 
need  to  restrict  A  to  be  real  and  symmetric.  If  there  is  a  unique  eigenvalue  X  of  maximum  absolute 
value  the  sequence  { x,- }  will  converge  to  an  eigenvector  y  corresponding  to  X.  The  ratios  of  corresponding 
nonzero  components  of  x  and  Ax  will  converge  to  X. 

We  shall  not  dwell  further  on  the  power  method.  Its  properties  are  well  known.  The  method 
can  be  modified  in  many  ways  so  as  to  speed  up  convergence.  It  is  closely  related  to  the  gradient 
method,  which  will  be  discussed  in  the  next  section. 

12.3.  The  Gradient  Method  [1] 

The  gradient  method  is  based  on  the  fact  that  if  the  equation  Ax~\x,  (x^^O),  holds,  then  X  is 
given  by  the  formula  X=/i(a;),  where 

.(x)=^^-  (1) 

Here  x*  denotes  the  transpose  of  x.  If  A  is  real  and  symmetric,  as  we  shall  assume,  the  critical  points 
of  nix)  are  eigenvectors  and  the  critical  values  are  eigenvalues.  In  particular  the  maximum  of  n{x) 
is  the  greatest  eigenvalue  and  the  minimum  of  n(x)  is  the  least  eigenvalue.  This  suggests  that  these 
extreme  eigenvalues  can  be  obtained  by  finding  the  maximum  and  minimum  of  n(x).  The  direction  of 
steepest  ascent  is  given  by  the  vector 

^=Ax—n(x)x,  (2) 

which  we  shall  call  the  gradient  of  n(x)  at  x,  or  more  simply  the  gradient  at  x.  It  is  clear  that  a  vector 
xy^O  is  an  eigenvector  if,  and  only  if,  ^  =  0.  Thus,  the  magnitude  of  ^  relative  to  the  magnitude  of  x 
can  be  taken  as  a  measure  of  the  deviation  of  x  from  an  eigenvector. 

In  order  to  obtain  the  maximum  of  ix{x)  by  use  of  the  gradient  method,  we  use  the  iteration 

3.i+i  =  Xi-{-ai^i         (i  =  0,l,2,.  .  .),  (3) 
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where  ^i  is  the  gradient  of  ti(x)  at  a;,.  It  can  be  shown  that  the  scalars  a,  can  be  chosen  so  that  the  in- 
equahty 

^i{x^+^)-n{x,)^c\^i\'  (4) 

holds,  where  c  is  a  positive  number  independent  of  i.  In  particular,  this  is  the  case  when  at  is  on  the 
range 

0<5<a,<^-5,  (5) 

where  M  is  the  spread  of  the  eigenvalues.  If  a,  is  chosen  so  that  (5)  holds,  then  the  sequence  {fi(xt)} 
will  converge  to  the  greatest  eigenvalue  Xn,ax  unless  the  initial  vector  Xo  is  orthogonal  to  the  eigenvectors 
corresponding  to  Xn.ax.  Moreover,  the  sequence  {x^]  will  converge  to  the  corresponding  eigenvector.  If 
ai  is  replaced  by  —a,  in  (3),  the  minimum  is  obtained  in  place  of  the  maximum.  In  this  event  the  in- 
equality (4)  takes  the  form 

fJi{Xi)  —  ti(Xi+i)>c\^i\\ 

There  are  three  methods  of  choosing  the  parameter  a^  which  have  proved  to  be  practical. 

The  first  method  consists  of  assigning  a  fixed  value  to  a,.  Normally  we  seek  to  select  a,  to  be  on  the 
range  (5).  Inasmuch  as  the  spread  M^a^ax— «min  is  not  known  beforehand,  one  might  expect  that  it 
would  be  difficult  to  make  a  suitable  choice  of  «<.  However,  this  is  not  the  case.  If  the  largest  eigenvalue 
is  sought  the  computer  adjusts  a^  so  that  tJi(Xi+i)> fiixi) .  If  at  any  step  this  inequality  fails  to  hold,  the 
computer  knows  that  at  is  too  large  by  at  least  the  factor  two  [1].  Thus,  a  suitable  choice  of  a  can  be 
determined  by  watching  the  growth  of  fx{x).  An  experienced  computer  will  vary  a  from  time  to  time  in 
order  to  accelerate  the  convergence.  We  have  found  that  interspersing  a  few  large  values  of  a  with  smaller 
ones  is  very  effective. 

The  second  method  is  to  select  a,-  at  the  ith.  stage  so  that  n{Xi+a^i)  is  a  maximum.  In  this  event  a^ 
is  given  by  the  formula 


ocr 


where 

s,=MaO-M(:c,),     ^^jl;]-  ■  (6) 

When  ti  is  small,  as  it  will  be  when  Xi  is  a  good  estimate  of  the  solution,  then  ai=\/\Si\  is  a  good  estimate 
of  the  optimum  value  of  at.  In  any  event,  it  is  an  over  estimate  of  this  optimum  value.  It  has  been  our 
experience  that  this  method  converges  too  slowly.  Normally  accelerations  are  needed.  We  have  found 
that  this  method  is  not  as  good  as  the  one  described  above. 

The  third  method  is  to  choose  a,-  to  be  a  fraction  of  the  optimum  value  described  in  the  preceding 
paragraph.  For  example,  one  can  choose  ai=^/\si\,  where  Sj  is  given  by  (6)  and  /3  is  a  fixed  number  on 
the  range  0</3<l.  In  particular,  the  values  /3=.7,  /3=.8,  /3=.9  have  been  found  to  be  effective  in 
selected  examples.  This  method  has  the  property  of  being  self-accelerating  in  the  sense  that  at  relatively 
frequent,  but  irregular,  intervals  large  corrections  are  made.  The  experiments  that  we  have  carried  out 
to  date  seem  to  indicate  that  this  method  is  superior  to  those  discussed  above,  including  the  power 
method. 

The  connection  between  the  gradient  method  and  the  power  method  can  be  seen  if  we  write  the 
iteration  (3)  in  the  form 

Xi+i=ai{A—Cir)Xi, 
where 

Ci  =  n{Xi) 

In  the  power  method,  Ci  is  held  fast.  In  the  gradient  method,  Ci  is  determined  at  each  stage.  It  is  clear 
that  a  wise  choice  of  c,  will  greatly  accelerate  the  process.  It  is  for  this  reason  that  the  gradient  method 
appears  to  be  preferable  to  the  power  method. 
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12.4.  A  Generalization  of  the  Gradient  Method  [2] 

The  gradient  method  described  in  the  previous  section  can  be  looked  upon  from  a  shghtly  different 
point  of  view.  Let  {x,Ax,.  .  .,A''~h}  be  the  class  of  vectors  that  are  linear  combinations  of 
X,  Ax,  .  .  .,  A^'^x.  In  the  gradient  method  the  vector  x,+i  is  chosen  in  the  subspace  {xi,Axi],  so  that  the 
inequality 

I  ^{Xi+,)-fx{Xi)>cm'  (7) 

holds,  where  c  is  a  small  positive  number  independent  of  i.     An  obvious  extension  of  this  procedure  is 

to  select  Xi+i  in  the  class  {xi,Axi,.  .  .,A''~'^Xi],  where  k  is  fixed.     One  keeps  the  condition  (7).     If  one 

seeks  the  minimum  eigenvalue  in  place  of  the  maximum,  one  uses  the  inequality 

1 

^l(xi)  —  ^JL(Xi+l)>c\^i\^ 

in  place  of  (7). 

The  difficulty  encountered  in  this  procedure  is  that  of  selecting  a  suitable  vector  Xi^i.     For  example, 
!   one  might  select  Xj+i  so  as  to  maximize  (or  minimize)  the  function  /x(x).     This  would  involve  solving  the 

characteristic  equation  Pk{\)  =  0  of  A  on  the  space  {xi,Axi,    .    .    .jA^'^Xi).     Methods  for  doing  this 

will  be  described  in  the  sections  that  follow. 
I  A  theoretical  discussion  of  the  method  just  described  can  be  found  in  a  recent  paper  by  Karush  [2]. 

12.5.  Iterative  Methods  of  Generating  the  Characteristic  Polynomial  [3,  4,  5]. 

j  The  present  section  will  be  devoted  to  describing  an  iterative  method  of  generating  the  characteristic 

!  polynomial  of  A.  For  convenience,  we  shall  assume  that  A  has  n  distinct  eigenvalues.  This  is  not  an 
essential  restriction  and  is  made  to  simplify  our  discussions.  The  method  we  shall  describe  is  a  generali- 
zation of  one  due  to  Lanczos  [3]. 

Let  -B  be  a  positive  symmetric  matrix  that  commutes  with  the  given  symmetric  matrix  A.  We 
suppose  that  a  vector  Xo  has  been  chosen  so  that  the  vectors  Xo,  Axq,  .  .  .,  A"~%  are  linearly  independent. 
We  select  Xi  so  that 

Xi=CLoAxo     OqXo,  (8) 

where  Oo  is  a  scale  factor,  and  bo  is  chosen  so  that  x*Bxo=0.     The  formula  for_6o  is 

^'='''    x*Bxo  ' 
Having  chosen  the  vectors  Xq,  .  .  .,  Xt  (i?^l),  we  select  Xi+i  by  the  formula 

Xi+i=aiAxi—biXi—Ci.iXi-i,  (9) 

where  a^  is  a  nonzero  scale  factor  and 

,  _      xfBAxi 
j,  ^'-""'xfEx, 

_      x*.iBAxi_  ai      x*Bxt 

j  The  vectors  Xo,  -  .  .,  Xn-i  obtained  in  this  manner  satisfy  the  relation 

I  x*Bxj=Q,        19^  j.  (11) 

This  can  be  proved  by  induction  (see  reference  [4]).  Wc  have  already  seen  that  x^Bxi^Qi.  Suppose 
this  relation  holds  when  i<ij<k.  It  is  easy  to  sec  from  the  definition  of  hk  and  c*_i  jthat  the  relation 
holds  with  i==k,  k—\,  and  j=k^\.     Suppose  now  that  0<i<^— 1.'    Then 

x*Bxk+i=aKxtBAxH—hiX*Brk—Ct_^x*Bxk.y 
=akxtBAxi 

=—  xlB{xi+i-\-hxi-VCi_iXi_^)  =  0. 
at 
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A  similar  argument  holds  for  i=0. 

From  the  definition  of  Xi  it  is  clear  that  Xi  is  of  the  form  Xi=Pi{A)xo,  where  P,(X)  is  a  polynomial  of 
degree  i  in  X.     It  is  clear  from  (11)  that  Xn=Pn(A)xo=0.     Hence 

Pn{A)x,  =  Pn{A)Pi(A)Xo  =  Pi{A)Pn{A)Xo  =  0. 

It  foUows  that  PniA)  =  0.     The  polynomial  P„(\)  is  accordingly  the  characteristic  polynomial  of  A. 
From  the  relations  (8)  and  (9)  it  follows  that  the  pohTiomials  Pi(X)  satisfy  the  relations 

Po(X)  =  l,         Pi(X)  =  aoX-6o,         P,+i(X)  =  (aiX-6,)P,(X)-c,_iP,_i(X).  (12) 

Similarly,  the  derivatives  of  Pi(X)  satisfy  the  relations 

P'M  =  0,         P'M=ao,         P;+x(X)  =  (a,X-&.)P;(X)-c,_iP;_,(X)  +  a,P,(X).  (13)i 

By  using  (12)  and  (13)  one  can  obtaui  the  values  of  P„(X)  and  P'„{\)  for  a  given  number  X  \\'ithout  obtain- 
ing an  explicit  formula  for  P„.     We  can  accordingly  apply  Newton's  formula 

^'~^~PM 

to  compute  the  zeros  of  P„,  that  is,  the  eigenvalues  of  A.     Having  found  an  eigenvalue  X,  a  corresponding) 
eigenvector  y  can  be  obtained  by  the  formula 

y^hoXo^  .  .  .  +/i,!_iX„_i, 
where 

^'^Wpi,         (^  =  0,l,...,n-l). 

The  case  B=I  is  the  one  discussed  by  Lanczos  [31.  From  a  theoretical  point  of  view  nothing  is 
gained  by  using  a  more  general  matrix  B.  This  follows  from  the  fact  that  the  vector  Xo=B''xo  will  gen- 
erate the  same  polynomials  (using  B=r)  as  those  given  above,  ^^^lether  or  not  there  is  a  computational, 
advantage  remains  to  be  seen. 

The  iteration  described  above  normally  can  be  put  in  a  somewhat  different  form.  We  consider  only^ 
the  case  B=I.     The  new  form  is  given  by  the  system 

Xi+i=aiXi'\-^iAyi  (14)' 

Here  /3j  and  8i  are  nonzero  scale  factors  and 

In  the  unusual  case  when  y^  Ayi=0  for  some  i<C.n—  1  this  system  fails.    It  is  not  difficult  to  show  that 
the  relations 

x*X}=^,    y*Ay}=^,        {i^j)  (15) 

hold.     Moreover,  upon  eliminating  the  y's  in  (14)  one  obtains  the  formulas 

Xi^^QAXQ  +  a^XQ 

X  i  j^i^=^  di  Ax  i —  OiXi — Ci—iXi-i,  yi^yj) , 
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,nd  its  derivative  P„  by  the  formulas 


inhere 

n  a  similar  mamier  it  can  be  seen  that  the  relations 

?/i  =  /5o7o^?/o+  (5o  +  ao7o)2/o 

yi+i=a[Ayi—b[yi—cl-iy{-i 
lold  where 

Thus,  it  is  seen  that  the  x's  and  the  y's  have  the  properties  described'at  the  beginning  of  this  section.    The 
haracteristic  polynomial  P„{\)  can  be  obtained  from  the  formulas 

Po=^o=l 
Qi+i=8iPi+i  -{-8iQi 

P'o=^Qo=o 

ionsequently,  Newton's  methods  can  be  applied  as  before  to  obtain  the  zeros  of  F„(X). 

The  methods  described  in  this  section  can  be  readily  generalized  to  the  case  in  which  A  is  not  real 
Ind  symmetric. 

12.6.  A  Block  Method 

We  shall  describe  briefly  a  method  that  has  been  used  successfully  on  a  matrix  which  was  too  large 

0  be  handled  by  the  gradient  method  with  the  equipment  on  hand.  We  shall  call  this  the  block  method, 
t  can  be  described  as  follows: 

(1)  Partition  x  into  two  parts,  x={y,z),  where  y  denotes  k  components  of  x  and  z  the  remaining 
—k  components  of  x.    The  function  /i(x)  described  in  section  12.3  is  a  function  iJi{y,z)  of  y  and  z. 

(2)  Holding  z  —  Zi  fast,  select  yi  and  a  scalar  a^  such  that  n{y,ocZx)  has  a  maximum  value.    If  a?^0, 

1  e  select  ?/i  and  ai  so  that  ai=l.    A  vector  Xi=^(yi,  aiZi)  is  obtained. 

;  (3)  Iterate  steps  (1)  and  (2),  using  a  new  partition  for  each  iteration.  In  each  case  the  components 
0  be  held  fast  are  assigned  the  values  of  the  corresponding  components  of  the  vector  Xi-i  determined 
•a  step  (2)  of  the  preceding  iteration. 

'  "When  this  procedure  is  carried  out  in  a  cyclic  manner,  a  sequence  of  vectors  {xi}  is  obtained  which 
;onverge,  if  suitably  normalized,  to  an  eigenvector  corresponding  to  the  highest  eigenvalue. 

The  procedure  just  described  is  analogous  to  that  given  in  section  12.4.  In  each  case  ix(x)  is  maxi- 
lized  (or  minimized)  successively  on  linear  subspaces  Bi,  B^,  .  .  .  oi  our  space.  In  this  section, 
be  subclass  Bi  is  determined  by  Xi  and  the  partition  of  x,  whereas  in  section  12.4  the  space  Bt  was  de- 
srmined  by  the  vectors  x<,  Axi,  .  .  .,  A'^'^Xi. 

The  method  just  described  appears  to  be  well  adapted  for  machine  computation. 

12.7.   The  Case  Ax  =  \Bx  [6] 

We  now  tm-n  to  the  problem  of  finding  a  solution  of  the  system 

Ax=\Bx,  (16) 
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where  A  and  B  are  ri-dimensional  square  matrices.  We  consider  the  problem  in  the  domain  of  complex 
numbers,  and  assume  that  B  is  nonsingular.  The  matrices  A  and  B  need  not  be  Hermitian.  If  X  is 
chosen  so  that  (16)  has  a  nonnull  solution  x,  then  X  is  an  eigenvalue  and  x  an  eigenvector. 

Let  C,  D,  K{a)  be  the  matrices  ■ 

C=B*HA,        D^B*HB,        K{a)  =  {A-aB)*H{A-<7B),  (17) 


where  fl"  is  a  positive  Hermitian  matrix.  It  is  clear  that  P  is  a  positive  Hermitian  matrix  and  that 
K{<t)  is  a  nonnegative  Hermitian  matrix.  In  fact,  K{a)  is  positive  whenever  A—cB  is  nonsingular, 
that  is,  if  <r  is  not  an  eigenvalue  of  the  system  (16). 

The  results  to  be  described  in  this  section  are  centered  around  the  two  functions 

x*K{a)x  x*Cx 

These  functions  are  connected  by  the  relation  \ 

I 

f{x,c)=f(x,Kx))  +  W-Kx)\'.  (19)' 

From  this  result  it  is  seen  that /(x,cr)  =  0  if,  and  only  if,  a=fx(x)  and  f(x,n(x))=0.  Consequently,  a 
vector  X  9^  0  and  a  number  X  is  a  solution  of  (19)  if,  and  only  if,/(a;,X)=0.  Moreover,  \=ijl(x).  Thus, 
a  vector  x  is  an  eigenvector  if,  and  only  ii,  f{x,n(x))=0.  For  an  eigenvector  x  the  formula  (19)  takes 
the  form 

f(x,a)  =  \a-,ji{x)\\  (20) 

Consider  now  a  fixed  number  <r  and  let  Vi  and  r2  be  the  greatest  and  the  least  nonnegative  numbers' 
such  that  rl<f(x,a)<rl.     For  an  eigenvector  x  we  have  by  (20)  the  relation 

ri<\(T—fi(x)\<r2. 

It  follows  that  there  is  no  eigenvalue  within  the  circle  Fi  of  radius  ri  about  a  and  no  eigenvalue  exterior 
to  the  circle  r2  of  radius  r2  about  cr.  By  constructing  the  circles  Fi  and  r2  for  various  values  of  c  one 
determines  domains  in  which  eigenvalues  lie  and  in  which  they  do  not  lie.  If  C*=C,  it  can  be  shown 
that  there  is  an  eigenvalue  on  each  of  the  circles  Fi  and  F2.  More  generally,  if  the  system  (16)  ha^ 
n-linearly  independent  eigenvectors,  it  is  always  possible  to  choose  H  so  that  there  is  an  eigenvalue  ori 
each  of  these  ch'cles. 

The  remarks  made  above  suggest  the  following  procedure  for  finding  eigenvalues  and  eigenvectors 
Select  a  number  ai  and  determine  Xi  such  that/(x,o-i)  has  a  minimum  value  at  Xi.  Select  0-3= ^(^^i))  ^.nc 
determine  X2  such  that /(x, 0-2)  has  a  minimum  value  at  Xz.  Having  chosen  Xi-i,  set  <7i=fi(Xi-i),  anc? 
select  Xi  so  that/(a*,ai)  attains  its  minimum  at  Xi.  This  procedure  will  determine  a  sequence  Xt  sucL 
that  iJ.(Xi)  will  converge  to  an  eigenvalue  whenever /(a; f,a-j)  tends  to  zero.  This  method  has  been  carriec 
out  successfully  in  a  special  case.  Whether  or  not  it  is  a  good  method  for  computation  is  yet  to  be 
determined. 
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13.  New  Results  in  the  Perturbation  Theory 
of  Eigenvalue  Problems 

I  F.  Rellich  1 

13.1.  Introduction 

!       Let  us  consider  a  self-adjoint  eigenvalue  problem  A(p=\(p,  where  A=A(€)  depends  analytically  on 
!,  perturbation  parameter:  ^(c)=^4o  +  e-4i+   .  .  .,  the  simplest  case  being  ^(e)=ylo+€^i-     One  expects 
hat  eigenvalues  and  eigenf unctions  of  the  disturbed  operator  ^(e)  will  depend  analytically  on  e  for 
mall  \e\  thus 
I  X(e)  =  X(«)  +  eX<i>+    .   .   .   ) 

}•  (1) 

^(e)  =  ^(0)  +  e9.^^>+    .    .    .    ) 

^he  computation  of  X^'^  <p'-^\  .  .  .  yields  the  successive  approximations  of  the  unknown  X(€),  ^(e). 

Example  1.     One  of  the  first  problems  treated  in  this  way  in  the  literature  is  Lord  Rayleigh's 
iroblem  of  the  vibrating  string  of  small  stiffness.     In  a  slightly  generalized  form  this  is 

-u"-{-q{x)u+€u""==\u,     O^a^^l,     u{0)=u"{S))=u{\)=u"{\)  =  Q. 

\  think  nobody  will  expect  that  compared  with  the  undisturbed  term  —u"-\-q^{x)u  the  perturbation 
vb""  is  small  enough  to  allow  a  series  of  the  kind  (1)  with  e?^0. 

I      Example  2.     In  quantum  mechanics  the  perturbation  method  was  first  used  by  E.  Schrodinger, 
'.'ho  treated  the  eigenvalue  equation  of  the  Stark  effect,  i.  e., 

I  2Z 

—  {u„^u,,^u^^)  —  —u  +  txu=\u,     —  oo<x,i7,2<+ °°,     r  =  {x'^  +  y--\-z-y''^ 

t 

Q  this  example,  it  is  perhaps  less  obvious  whether  or  not  the  perturbation  term  txu  is  small  enough 

for  small  \t\)  as  to  guarantee  the  development  (1).     But  from  the  behavior  of  the  potential  energy 

-2Z/r4-er  one  may  conjecture  that  for  ej^O  the  problem  has  a  continuous  spectrum  without  point  eigen- 

alues  and  that  means  that  (1)  is  not  true,  at  least  not  in  the  usual  sense. 

Example  3.     The  wave  equation  of  the  nonharmonic  oscillator  is  given  by 

1  —u"  +  {x'^-\-ex'^)u=\u,     —  00  <j;<  oo 


..vith  an  additional  boundary  condition  at  a:=+  c»  and  x=—  ^  in  the  case  e<0  in  order  to  have  a  self- 
djoint  problem).  This  problem  has  a  pure  point-spectrum  for  every  (real)  value  of  e.  But  for  c^O 
ad  for  e<0  the  spectra  of  the  problem  are  quite  different.     In  the  first  case  we  have  lim  X„(e)  =  +  »  ; 

i  n— »oo 

|i  the  second  case  the  spectrum  consists  of  two  sequences  X„(€),  n„{e)  of  eigenvalues  with  lim  X„(e)  =  +  <»  ^ 

n— •<» 
|m  Mn(e)  =  —  0°  . 

>-»oo 

This  difference  is  well  known  for  the  mechanical  analogue,  i.  e.,  DuflTmg's  vibration  problem  with  the 
onlinear  restoring  force  —2x—AiX^).  Again  it  is  doubtful  whether  the  perturbation  ex'^u  is  small 
iiougli  as  to  expect  convergent  power  series  (1). 

In  each  of  these  examples  it  is  easy  to  compute  X^'>  by  the  well-known  perturbation  procedure. 
lut  it  is  not  easy  to  say  what  is  the  sense  of  X^^^  +  cX'*^  as  an  approximation,  if  there  is  a  sense  in  it  at 
ll. 
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The  question,  what  is  a  "small"  perturbation,  is  answered  in  a  satisfactory  way  for  several  types 
of  eigenvalue  problems.  The  answer  is  not  a  unique  one.  A  perturbation  can  be  small  with  respect 
to  tlie  isolated  point  eigenvalues  of  the  undisturbed  operator  but  not  small  with  respect  to  its  continuous 
spectrum.  Certainly  there  are  a  lot  of  reasonable  questions  for  which  the  perturbation  theory  as  devel- , 
oped  until  now  gives  no  answer,  but,  in  spite  of  that,  I  think  the  perturbation  theory  of  eigenvalue 
problems  is  in  a  better  situation  than  its  elder  brother,  the  perturbation  theory  of  classical  mechanics, 
which,  e.  g.,  until  now  has  no  other  characterization  for  small  perturbations  than  smallness  of  the  pertur- 
bation parameter. 

I  gave  a  survey  of  the  known  mathematical  results  in.  the  perturbation  theory  of  eigenvalue  problems 
at  the  International  Congress  of  Mathematicians  in  Cambridge  (Mass.)  last  year.^  Since  that  time  two 
new  results  were  obtained,  which  I  will  describe  in  what  follows.     Both  of  them  are  due  to  E.  Heinz.^_ 

i 

13.2.  The  Dependence  on  the  Perturbation  Parameter  for  the 

Resolution  of  the  Identity 

To  describe  the  spectrum  of  a  self-adjoint  operator  Hilbert  mtroduced  the  resolution  of  identity 
^^   _  00  <X<  oo  .     Instead  of  a  definition,  I  give  an  example:    Let  ^ 

Au=-u",     -l<x<l,     ui-l)=u{l),     u'{-l)=u'{l). 
Then  the  resolution  of  the  identity  of  this  operator  is  given  by 

E^u  =  JZi^n,u)<p.,     K  =  nVIP,     ^„  =  (2/)-'/V"'^/',     {<pn,u)==r':FMu{x)dx,     n  =  0,±l±2,  .  .  . .    I 
x„<x  ^-^  || 

If  A  depends  on  e  we  shall  have  Ex=Ex(e) .  Now  we  make  the  following  assumptions  about  A  =  A{e)  :■ 
In  a  subspace  A  of  a  Hilbert  space  H  the  operator  Ao  is  self -adjoint  and  Ai,  A^,  .  .  .  are  Hermitian  oper- 
ators in  A.  For  each  u  of  A  we  require  ^(e)w=Aw+€^iW+  •  .  .,  where  the  power  series  at  the  righthand 
side  converges  in  the  sense  given  by  the  norm  \\u\\  =  {u,uy'^  defined  in  the  Hilbert  space.  Then  it  can  be  proved 
that  two  constants  a  and  k  exist  such  that 

11^,1^11  ^a^''(!|Aow||  +  !|u||),  i'  =  0,l,2,...  (2). 

for  all  u  of  A  and  that  for  small  e  the  operator  Aie)  is  self -adjoint  in  A. 

For  differential  operators,  however,  it  cannot  always  easily  be  decided  whether  or  not  an  operator 
Ai  (or  A2,  A3,  .  .  .)  can  be  defined  as  an  Hermitian  operator  in  the  subspace  A  in  which  Ao  is  self-adjoint 
It  is  therefore  more  convenient  to  suppose  that  Ao  is  essentially  self-adjoint ''  in  a  subspace  D  of  A 
which,  in  general,  will  be  smaller  than  A  and  will  allow  to  see  immediately  that  Ai,  A2,  .   .   .  can  b( 
defined  as  Hermitian  operators.    Now  the  existence  of  two  constants  a,  k  for  which  the  inequalities  (2) 
hold  can  no  longer  be  proved  but  are  an  additional  assumption.    With  the  aid  of  this  assumption  one' 
can  prov-e  that  ^(€)=.i4o+€^i  +  .  .  .  defined  in  D  can  be  extended  by  closure  to  an  operator  self-adjoint 
in  A^D.     Thus  the  first  definition  of  A(e)  in  A  is  regained  and  we  shall  use  it  further  in  this  section. 
(The  assumptions  of  this  definition  are  not  fulfilled  in  our  tlu-ee  examples  of  the  introduction  (with 
A2=Az=.  .   .  =  0)  but  they  are  fulfilled  for  the  second  and  the  third  example  if  the  independent  variables 
are  restricted  to  a  "finite  box")- 

The  question  arises  whether  such  an  operator  A(e)=Ao-\-eAi-\- .  .  .,  self-adjoint  in  A  for  small  \e\ 
has  an£'x(€)  that  is  regular  with  respect  to  e,  Ex(e)=Ei°' -^eEi'' + .  .  .  (The  operators  Ex(e),Ei°\El 
.  .  .  being  bounded  operators,  the  meaning  of  this  series  is  obvious). 

B.  V.  Nagy  ®  proved  the  following  important  theorem.  If  the  spectrum  of  the  undisturbed  operator 
Ao  is  empty  in  the  two  intervals  Xq— f/<X<Xo+c?«^c?  Mo— '^^<^<mo+c?  then  E\^(e)  —  E^^(e)  =  Po-^- eI\-\- .  .  .,s 

2  F.  Rellich,  Stbrungstheoric  der  Rpektralzerleguns,  T'roc.  Intern.  Congr.  Math.  Cambridge  (Mass.)  (1950). 

3  E.  Heinz,  BeitrSge  zur  Ptorungstheorie  der  Spektralzerlegung,  Math.  Ann.  123,  415-438  (1951). 

♦  .4ois  es.'sentially  self-adjoint  in  D  if  it  is  self-adjoint  in  .4£Z>and  if  to  each  u  of  A  a  sequence  Un  of  D  can  be  found  with  ||tt— «„||— >0  and  ||,'lo(u— «n)||-*.' 
n—*o=,  Ao  in  A  is  called  the  closure  of  ylo  in  D. 

*  B.  de  Sz.  Nagy,  Perturbations  des  transformations  autoadjointes  des  I'espace  de  Hilbert,  Comment.  Math.  Helv.  19,  347-366  (1946). 
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ohere  Pq,  Pi,  ■  •  •  are  bounded  operators  and  the  series  is  convergent  for  small  values  of  \e\.  E,  Heinz 
liscovered  that  even  E\^(i)  =  Qo-{-eQ i-'r.  .  .  (with  bounded  Qo,  Qi,  .  .  .  and  converging  series)  is  true 
provided  that  the  spectrum  of  Aq  is  empty  in  Xq— c?<CX<^Xo+c?.  This  theorem  of  Heinz  contains  the 
,heorem  of  v.  Nag}',  but  it  is  much  further  reaching.  I  shall  not  give  the  proof  here,  but  in  section  13.4 
;  shall  prove  an  inequality  that  is  used  by  Heinz  as  an  essential  tool  in  his  proof. 

13.3.  Convergent  Sequences  of  Operators 

Several  j^ears  ago  I  proved  the  following  theorem.  Let  A  in  A  and  A^^^  in  ^4^"^,  ?i=l,  2,  .  .  .be 
elf -adjoint  operators,  A,  ^4^"^  subspaces  of  a  Hilbert  space  H.  Let  the  subspace  D  be  contained  in  each 
f  the  spaces  A,  A^^\  A'-^\  .  .  .  and  let  A  in  D  be  essentially  self -adjoint.     Fivally,  lim  ||(yl^"^— yl)ii||  =  0 

\or  each  u  of  D.     If  Xq  is  not  a  point-eigenvalue  of  A,  then  lim  WiE^^^  —E\^)f\\=0  for  all  f  of  H,  where 

71— »00 

\i^^w  j-gf>p  2?x  are  the  resolutions  of  identity  of  A'-"''  resp.  A.  This  theorem  is  rather  general.  It  even 
[covers  Rayleigh's  example  mentioned  above.  Indeed,  call  D  the  space  of  all  functions  u{x)  with  con- 
inuous  fourth  derivatives  in  O^a-^  1  and  u{0)  =  u"{0)=u{l)=u''{l)  =  0.  The  operator  —(d^/dx^)-{-q(x) 
s  essentially  self-adjoint  in  D,  we  call  A  its  closure.  The  operator  —(d^/dx'^)-\-q(:x)-\renid'^ldx*)  with 
r,9^0,  lim  e„  =  0,  is  again  essentially  self-adjoint  in  D,  we  call  its  closure  /1^"\    Obviously 


\\{A^'^^-A)u\\  =  U(^£\^^,\'dxy-^0,n 


^hus  lim  WiEx^"''  —E\)f\\=0  if  Xq  is  no  eigenvalue  of  A.    Because  of  the  fact  that  this  limit-relation 


"0 
n—Ko 


n  general  is  not  true  uniformly  for  all/  of  H  with  ||/||  ^1  the  statement  established  by  this  relation 
s  a  very  weak  one. 

In  order  to  obtain  uniform  convergence  we  replace  the  condition 

lim  1|(/1("^-/1)m||  =  0,  ueD 

y  the  stronger  condition 

\\iA^">-A)u\\^Vn{\\u\\  +  \\Au\\} 

or  all  u  of  D,  where  r;„  is  a  sequence  of  numbers  with  lim  77„=0.     (In  Rayleigh's  example  this  stronger 

n— »oo 

ondition  is  obviously  not  satisfied). 

The  question  is  whether  this  stronger  condition  guarantees  lim  \\(Ei"^^  —  Ex^)f\\  =  0,  uniformly  for  all 

n-»oo 

'  of  H  with  Il/ll  ^  1  in  the  case  that  the  spectrum  of  A  is  empty  in  an  interval  Xq— (/<X<Xo+(/.  Heinz 
ave  the  first  proof  for  this  statement.  Again  an  important  tool  of  this  proof  is  the  inequality  of  the  next 
ection. 

13.4.  Heinz's  inequality 

Let  D  be  a  subspace  of  a  Hilbert  space,  A  in  D  an  essentially  self -adjoint  operator  and  Q  in  D  an 
lermitian  operator.     Further,  0  ^  {u,Au)  and  (Qu,  Qu)  g  {Au,  Au)  for  all  u  of  D.     Then 

I  {u,  Qu)  I  ^  {u,Au)        for  all  u  of  D.  (3 ) 

;'his  inequality  is  the  simplest  among  much  more  general  inequalities  obtained  by  Heinz,^  but  it  is  sulh- 
;ient  for  the  purposes  mentioned  in  section  13.2  and  13.3.  In  Avliat  follows  we  shall  give  a  simple  proof 
f  (3).     From  (3)  follows  \{u,Qv)\^^  iu,Au)iv,Av)  for  all  u,v  of  D  in  the  well  known  way. 

In  the  special  case  of  an  w-dimensional  Hilbert  space  and  a  nonnegative  Q,  i.  e.,  {u,Qu)^  0  for  u 
,1  D,  the  inequality  is  known.  In  fact,  if  wo  put  R=Q\  S=A\  we  have  {u,Ru)S(u,Sa),  and  thus  the 
licquality  {u,W^'^u)  S{u,S^'^u)  is  well  known  from  Loewner's  theory  of  monotone  functions  of  matrices. 


•  See  footnote  3. 

i)7 


A  simple  proof  ^  (for  the  n-dimensional  space)  runs  as  follows.  Let  X  be  an  eigenvalue  of  the  Hermitian 
operator  u4—Q,  thus  (A—Q)<p=\(p,\\<p\\  =  l.     Hence 

H{A+Q)<p,<p)  =  ((A  +  Q)cp,(A-Q)<p)=:{A^,A<p)-(Q<p,Q<p)  +  (Q<p,A<p)-(A<p,Q<p)' 

Taking  the  real  part  of  this  equation,  we  get 

\aA-{-Q)<p,<p)  =  (A<p,A<p)-(Q<P,Q<p)^0. 

Adding  \{(A—Q)<p,(p)^  X^,  we  have  2\(A(p,(p)^  y,  and  therefore  X^  0  because  of  (A<p,<p)^  0.  However 
if  every  eigenvalue  of  A—Q  is  positive,  or  zero,  then  (u,Au)^  (u,Q,u).  Replacing  Q  by  —Q,  we  get 
{u,Au)^  —{u,Qu);  hence  {u,Au)^  \{u,Qu)\. 

With  some  carefulness  the  idea  of  this  proof  can  be  used  in  the  Hilbert  space  of  not  finite  dimension 
The  closure  of  ^  in  Z>  is  a  self-adjoint  operator  A  in  A^D.  Because  of  (Qu,Qu)^(Au,Au) 
the  operator  Q  can  be  extended  to  an  Hermitian  operator  in  A.  The  operator  A—Q  is  in  general  not 
self-adjoint  in  A.  Therefore,  we  introduce  A—kQ,  where  A:  is  a  fixed  number,  — 1</:<1.  This 
operator  is  self-adjoint  in  A.  Indeed  A—kQ—i=(l—kR){A—i)  with  R—Q(A—i)~^.  The  operatoi 
R  is  defined  in  H  and  we  have  ||/?/||  =  ||(^(^-'i)-y||  or,  with  u={A-i)-'f,{A-i)u=f,  \\Qu\\^\\Au\. 
^11/11,  hence  ||i?/||  ^  ||/||.  Therefore,  (A-kQ-i)-'=(A-i)-'il-\-kR-^k'R'+  .  .  .  )  is  a  bounded 
operator.     The  same  being  true  for  {A—kQ-\-i)~^  we  have  proved,  that  A—kQ  is  self-adjoint  in  A. 

Let  X  be  a  point  of  the  spectrum  of  A—kQ.  Then  the  spectral  space  of  this  operator  belonging  tc 
the  spectral  interval  X  — 1/n  to  X  +  l/w,  n=l,  2,  .  .  .,  is  not  empty;  it  contains  at  least  one  element  (pn, 
thus 

It  follows 


I 


{A—kQ)(p„  —  \<pn  =  i>}n     if     00„=  I         {n  —  \)dEy,(Pn. 


We  find 

H{A-^m<P.,^n)^{{A^kQ)<pn,{A-kQ)<p.)-{{A^kQ)^n,<^n) 

=  {A<Pn,A<Pn)-k\Q.Vn,Q<Pn)-^^{{{A^-kQ)^n,0^n)] 

^   -\{{A-{-kQ)^n,0^n)\ 

^   -\\{A^-kQ)^n\\-\\o:n\V 

Adding 

\{{A-kQ)<Pn,^n)=y^Ho^n,<Pn)^  \'-\\\    \Wn\\, 

we  have 

2X(^^„,^„)^X2-|X|  \M\-\\{A^kQ)^n\\-\Wn\V     , 


We  have  lim  ||co„I|  =  0,  and  from  (yl-^-(?)^„=X^„+co„  obviously  \\{A—kQ)ipn\\^C,  where  C  does  no^ 
depend  on  n.     Hence  ||^^„||^|^|  \\Q^n\\  +  C^\k\  ||.4^„|H-C, 

\\A^n\\^CI{l-\k\),  |l(^+^())^„||^|!^^„||  +  |Ar|  il^^nll^  j±||-|  C. 

l  —  \k\ 

If  X  7^0  it  is  therefore  possible  to  choose  a  number  N  such  that 

-|X|  \\o^n\\-\\{A^kQ)^n\\  \M\^  -XV2         for  n>iV. 

For  these  n  we  have  2X(^^„,<^„)^  X72,  and  from  (^^„,<p„)^  0  it  follows  that  X>0.  At  any  rate  X^  0 
No  point  X  of  the  spectrum  of  A—kQ,  is  negative.  This  means  {{A—kQ)u,u)^  0  for  all  u  of  A.  This 
is  true  for  k  with  — 1<^<1.  The  limit  Ar->1  resp.  k->—l  yields  {{A—Q)u,u)^  0  resp.  {{A-{-Q)u,u)^  0 
which  is  the  affirmed  inequality  \{u,Qv)\^  {u,Au). 

'  I  owe  this  to  F.  A.  Flcken. 
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Heinz's  iueqiialily  is  nonelementary  in  the  sense  that  it  is  no  longer  true  if  the  {issimii)tion  "A 
essentially  self-adjoint  in  D"  is  replaced  by  the  weaker  assumption  "A  Hermitian  in  D."  This  is 
jhown  by  the  following  example.  We  choose  as  Hilbert  space  the  set  of  functions  ^<(a:)  defined  in  0<x<  oo 
,vith  finite 


(         |u|^c?2' ^=11^11,         (y, 'u)=        v{x)u{x)d: 


ind  as  D  the  subspace  of  all  u(x)  with  continuous  second  derivatives  in  0<Cx<C  °°  vanishing  identically 
in  neighborhoods  of  2=0  and  x=  oo .  We  define  Au=—u"  and  Qu  =  jX~^u  for  u  in  D.  The  operator 
1.4,  as  well  as  Q,  is  Hermitian  in  D.  The  operator  Q  in  D  is  even  essentially  self -ad  joint  with  Q^O 
md  the  closure  of  .1  in  D  is  an  Hermitian  operator  for  which  v.  Neumann's  deficiency-indices  are 
II,  1. 

By  partial  integration  and  Schwarz's  inequality,  one  proves 


'hence  (Qu,  Qu)  ^  (Au,  Au). 
The  identity 


^   r  x-'\u\'dx^  r\u"\'dx; 
It)  Jo  Jo 


u 


dx, 


which  holds  for  u  in  D  shows  (u,  Au)  ^  1/3  (w,  Qu)  ^0  for  u  of  D.  In  order  to  get  a  contradiction  to  (3) 
we  only  have  to  find  a  function  u  of  D  for  which  (u,  Qu)>0  and  l>(w,  Au)j{u,  Qu).  Define  a  sequence 
of  functions  u„(x)  (which  are  not  functions  of  D)  b}' 


Unix) 


f  0,  0^x^1/271 

(x-l/2n)2^,      l/2n^x^lln 


J/2 


{2n—x)lyln, 


l/n^x^n 
n^x^2n 
2n  <  a:  <  00 , 


For  these  functions  compute 


T(u„)=rWdxirx-'\un\'dx^^-^ for     n-^00. 

Jo  /Jo  ^-1-0(1) 

Jl/n  X 

|Thus  lim  T{u„)^l,  and  one  can  choose  a  number  7?o  for  which  T(u„^)<^^.    It  is  easy  to  find  a  function 

n— >oo 

Ur^OoiD  forwhich|r(w„)  — r(u)|<e  mth  prescribed  €>0.     We  choose  a  u  oi  D  for  which  T(uXh- 


From 


^  '     4  (v,  Qu) 


we  have  Y>\{u,  Au)/iu,  Qu),  \>(u,  Au)/(u,  Qu). 


JM) 


I 


14.  Bounds  for  Characteristic  Roots  of  Matrices 

n  Alfred  Brauer^ 

For  many  applications  in  different  fields  it  is  of  importance  to  determine  the  characteristic  roots  of  a 
Tiatrix  approximately  or  to  find  at  least  small  regions  that  contain  the  characteristic  roots.  Theoreti- 
:ally,  this  is  a  very  simple  problem.  Because  the  characteristic  equation  of  a  matrix  of  order  n  is  an 
ilgebraic  equation  of  degree  n,  any  method  for  the  approximation  of  the  roots  of  an  algebraic  equation 
:>ould  be  used. 

But  in  practice,  this  is  only  possible  for  very  small  values  of  n,  since  otherwise  the  computation  of  the 
:oefficients  of  the  characteristic  equation  is  very  tedious.  Therefore,  it  is  of  importance  to  have  methods 
that  give  approximations  for  the  roots  without  having  to  compute  the  coefficients  of  the  equation. 

A  large  number  of  papers  have  been  published  on  this  subject.  I  do  not  want  to  give  a  historical 
account  here.     This  is  done  in  three  papers  by  E.  T.  Bro\me  [1],  Olga  Taussky  [2],  and  W.  V.  Parker  [3]. 

Let  A=(a^\)  be  an  arbitrary  square  matrix  of  order  n.     We  set 

P.=  i:M         {K=l,2,...,n).  (1) 

x=i 

It  is  well  known  (see,  for  instance,  [4])  that  each  characteristic  root  of  ^1  lies  in  the  interior  or  on  the 
boundar}'  of  at  least  one  of  the  n  circles 

\z-a..\^P.         (K=l,2,...,n).  (2) 

A  few  years  ago,  I  improved  this  result  as  follows  [5]. 

Theorem  1.  Each  characteristic  root  of  A  must  lie  in  the  interior  or  on  the  boundary  oj  at  least  one  oj 
the  n(n  —  l)/2  ovals  of  Cassini 

l2-a„||s-axxI^P,Fx         ('^,X=1,2,  .  .  .,n;  kj^X).  (3) 

Every  point  of  these  ovals  lies  in  the  interior  or  on  the  boundary  of  at  least  one  of  the  circles  (2). 
I  want  to  improve  theorem  1  a  little  further.     The  formulation  of  the  following  theorem  is  somewhat 
more  difficult,  but  its  application  is  as  simple  as  that  of  theorem  1. 

Theorem  2  [6].     Let  A=  (aa)  he  a  square  matrix  of  order  n  and  P«  he  defined  by  (1).     Set 

P.\=\a,x\P\+\ax.\iPK—\a,\\)-\-^\a,,axy\+^\a,,ax^-{-a,^axy\,  (4) 

where  k,  X,  /i,  and  v  run  from  1  to  n,  and  where  kj^\;  ht^k,  X;  vt^k,  X.      Then  each  characteristic  root  u  of 
A  lies  in  the  interior  or  on  the  boundary  of  at  least  one  of  the  n  (n— 1)/2  ovals  of  Cassini 

|2— a„||2— ax\|^P«x     (k,  X=l,  2,  .  .  .,  7?;  /c5^X).  (5) 

It  is  obvious  that 

P,x^PJ\.  (6) 

We  have  the  equality  sign  in  (6)  if,  and  only  if, 

'^\a^yax^-{-a,^ax,\  =  ^\a^,ai^\-\-^\a^^a).,\. 

y<M  y<ti  »<M 
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Prooj  oj  theorem  2.     Since  w  is  a  characteristic  root,  the  system  of  Hnear  equations 

n 

'^a,xX\=oiX^         {k=1,2,.  .  .,n) 
x=i 

has  a  nontrivial  solution  Xi,  Xi,  .  .  .,  Xn-     Assume  that 

|a:,|^|Ts|^max|x,|         (v=l,2,.  .  .  ,n;     V9^r,s) 
We  consider  the  rth  and  the  sth  of  the  equations  (7) 


(7 


(8 


n 


TO 


(9 


(10 


Uxs=0,  then  Xy=0  for  every  V9^r.     It  follows  from  (9)  that  o}=arT  since  XrT^O.     This  proves  theo 

rem  2  if  x,=0.  , 

Assume  now  that  Xs?^0.    Multiplying  the  equations  (9)  and  (10)  we  obtain  i 

{o)—arr){co—as,)XrXs=(^ar,x,){Y]as^x^) 

■O'TsXs  y   I  ttsgXu.'T' (IstXt    y   ,   drvX-v]       X    i   drvO'syX^'V  ^_j    ((lrf(lfii-\~Qrii(lsf)XvX«' 


li^S 


v9^r,s 


V9^r,s 


hence  by  (1),  (4),  and  (8) 


P7^r,s 


This  proves  theorem  2. 

All  the  results  that  I  obtained  in  earlier  papers  [5,  7]  for  the  ovals  (3)  can  be  extended  to  the  ovals  (5)i 
Only  some  of  these  theorems  will  be  mentioned. 

Theorem  3.  Let  fi(y) ,  f2(y) ,  .  .  .,  fn  (y)  be  arbitrary  polynomials  and  A  a  square  matrix  of  order  v 
Let  A*  be  the  matrix  which  has  as  vth  column  the  elements  of  the  vth  column  of  the  matrix  f^  {A)  for  v=\,  2 
.  .  .,71.  Denote  the  elements  of  A*  by  aX  and  the  sums  corresponding  to  the  sums  (1)  and  (4)  by  P*  and  P* 
Then  each  characteristic  root  co  of  A  satis^es  at  least  one  of  the  n(n  —  l)/2  inequalities 

l/«M-«.*JI/x(co)-ax*xl^P.*x^P*Px*  («,  X-1,  2,  .   .   .,n;K^\).  ^ 

If  we  use  this  theorem  (see  [7])  for  the  matrix 


"9 

0 

0 

1 

l" 

1 

2 

2 

1 

0 

0 

1 

3 

2 

0 

0 

1 

1 

1 

0 

J 

1 

1 

2 

1 

and  take  for  the  polynomials /^(^z)  suitable  cubic  polynomials,  then  we  obtain  for  the  absolutely  greatest 
characteristic  root  co  the  inequality  9.1686<a;<9.1851  while  actually  9.17267<co<9.17268.  Here  th( 
error  for  the  lower  bound  is  less  than  0.05  percent. 

A  matrix  with  a  dominant  main  diagonal  is  a  matrix  for  which  ! 


„|>P,         ('c=l,2,.  .  .,n). 


(ii; 
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Minkowski  [8]  has  already  considered  such  matrices  in  the  special  case  in  which  all  the  main  diagonal 
elements  are  positive,  and  all  the  other  elements  are  negative.  He  proved  that  their  determinants  are 
positive.  The  same  holds  for  the  determinants  of  all  matrices  which  satisfy  (11).  This  generalization 
)f  Minl^owski's  theorem  follows  at  once  from  the  fact  that  the  characteristic  roots  lie  in  the  interior  or 
)n  the  boundary  of  the  circles  (2).  These  circles  do  not  intersect  the  imaginary  axis  if  (11)  holds.  Hence 
10  characteristic  root  equals  0,  and  the  determinant,  the  product  of  the  characteristic  roots,  cannot  vanish. 

Similarly,  it  follows  from  theorem  1  that  the  determinant  cannot  vanish  if 

I 

k.axx|>P«Px         (/c,  X=l,2,.  .  .,n;     k9^\)  (12) 

lolds  instead  of  (11).  When  I  published  this  theorem  f5],  I  did  not  know  a  theorem  of  Ostrowski  [9] 
Tom  which  the  same  result  follows  without  using  the  characteristic  roots.  If  Ostrowski's  theorem  is 
ipplied  to  the  characteristic  determinant  of  an  arbitrary  square  matrix,  then  theorem  1  can  be  obtained. 
The  inequalities  (12)  imply  that  (11)  holds  for  all  but  one  k. 

We  now  obtain  from  theorem  2 

Theorem  4  [6].     Assume  that 

|a,,axx|>Pa         {k,\=\,2,  .  .  . ,  n;  k9^\).  (13) 

Then  the  determinant  of  A  does  not  vanish.    If  moreover  akk^'O  for  k=1,2,  .  .  .,n,  and  if  the  characteristic 
iquation  has  real  coefficients,  then  the  determinant  of  A  is  positive. 
Further,  theorem  4  generalizes  Minkowski's  theorem.    It  foUows,  for  instance,  that  the  determinant 

a         1  1         1         1 

-1         a  1-1  -1 

D=       1-1  a         1  — 1  >0 

1      -1  1         a         1 

1—1  1         1         a 

•or  a>12*  since  the  inequalities  (13)  are  satisfied.  If  moreover  a<4,  then  each  element  of  the  main 
diagonal  is  smaller  than  the  sum  of  the  absolute  values  of  the  other  elements  of  the  same  row. 
i  Denote  by  ai,  a2,  .  .  .,  a^  any  k  different  integers  of  the  set  1,  2,  .  .  .,  n  and  by  /3i,  ^2,  ■  ■  -,  ^n-k 
the  remaining  n—k  integers  of  this  set.  I  call  a  square  matrix  of  order  n  reduced  if  it  is  possible  to  find 
1  set  of  integers  ai,  a2,  .  .  .,  a*  such  that  all  the  elements  of  the  matrix  vanish  which  are  in  common 
ho  the  rows  aj,  a2,  ...,«*  and  the  columns  /8i,  02,  ■  ■  ■,  0n~k-     Otherwise  I  call  the  matrix  unreduced. 

G.  Frobenius  [10]  uses  the  words  "zerfallend"  or  "zerlegbar"  for  such  matrices;  V.  Romanovsky 
,11]  the  word  "decomposable." 

I  O.  Taussky  [12]  proved  that  a  characteristic  root  of  an  unreduced  matrix  either  lies  in  the  interior 
bf  at  least  one  of  the  circles  (2)  or  on  the  boundary  of  each  of  them. 

This  theorem  can  be  extended  as  follows: 

Theorem  5  [13].     A  characteristic  root  of  an  unreduced  matrix  either  lies  in  the  interior  of  at  least  one 
)/  the  n{n—l)/2  ovals  (5)  or  on  the  boundary  of  each  of  them. 

Using  the  fact  that  the  roots  of  an  algebraic  equation  change  continuously  if  the  coefficients  are 
changed  continuously,  we  can  prove  the  following  result. 

For  each  k  we  consider  the  n—l  ovals 

|2-o**||2-axx|^P*x         (X=l,2,  .  .  .,n;\9^k)  (14) 

ind  take  of  each  of  these  ovals  only  that  simply  connected  region  bounded  by  (14)  which  contains  the 
:>oint  akk.     Wc  denote  the  closed  region  formed  by  these  n  simply  connected  regions  by  //*. 

Theorem  6  [Q].     IJ  one  of  the  regions  Ih  has  no  point  in  common  unth  the  region  formed  by  all  the  other 
f?x,  then  Hk  contains  one  and  only  one  characteristic  root  of  A. 

209431— 5.". 9  ^^^ 


A  square  matrix  A={a,^)  of  order  n  is  called  stochastic  if  all  the  elements  are  nonnegative  and  if 

X)aa=l  U=l,2,  ...n).  (15; 

x=i 

The  properties  of  the  characteristic  roots  of  stochastic  matrices  are  of  importance  in  the  theory  o] 

stochastic  processes. 

It  is  well  knowTi  that  all  the  characteristic  roots  of  a  stochastic  matrix  lie  in  the  interior  or  on  the 
boundary  of  the  unit  circle.  The  point  z=l  is  always  a  characteristic  root.  R.  v.  Mises  [20]  pointed 
out  that  the  results  of  G.  Frobenius  [14,  15,  10]  on  matrices  with  positive  and  nonnegative  elements 
can  be  used  for  stochastic  matrices  and  V.  Romanovsky  [11]  formulated  these  results  for  stochastic 
matrices. 

In  particular,  the  following  theorem  holds.  If  A  is  unreduced,  then  2=1  is  a  simple  characteristic 
root.  No  other  point  on  the  boundary  of  the  unit  circle  can  be  a  characteristic  root  unless  all  the  elements 
of  the  main  diagonal  vanish. 

N.  Dmitriev  and  E.  Dynkin  [16]  proved  that  no  characteristic  root  of  a  stochastic  matrix  of  ordei 
less  than  or  equal  to  n  can  lie  in  the  interior  of  the  segments  bounded  by  the  unit  circle  and  the  chords 
joinmg  the  point  2=1  with  the  points  2=6^'^''"  and  2=e"^'^*'".  In  a  second  paper  [17],  they  generahzed 
this  result. 

Let  an  be  the  smallest  element  of  the  main  diagonal  of  a  stochastic  matrix.  M.  Frechet  [18,  19] 
proved  that  all  the  characteristic  roots  lie  in  the  interior  or  on  the  boundary  of  the  circle 

\z—au\^l—(iii'  (16) 

Moreover,  without  using  the  results  of  Frobenius,  he  proved  that  no  point  different  from  1  on  the  bound- 
ary of  the  unit  circle  can  be  a  characteristic  root  unless  at  least  two  of  the  elements  of  the  main  diagonal 
vanish. 

Using  theorem  1  we  can  improve  Frechet's  theorem  as  follows: 

Theorem  7  [13].  Ij  an  and  ajj  are  the  smallest  elements  of  the  main  diagonal  of  a  stochastic  matrix,  then 
all  the  characteristic  roots  lie  in  the  interior  or  on  the  bovndary  of  the  oval  of  Cassini 

\z—au\  \z—ajj\-^{l-aii){l—aj^,  (17) 

The  proof  can  be  obtained  by  elementary  geometry  by  showing  that  each  point  of  the  ovals  (3)  lies 
in  the  interior  or  on  the  boundary  of  the  oval  (17). 

If  auT^ajj,  then  the  oval  (17)  lies  in  the  interior  of  the  circle  (16).  The  boundaries  of  both  curver' 
have  only  the  point  2=1  in  common.  Hence  (17)  gives  a  better  bound  for  the  nontrivial  characteristic 
roots,  that  is,  the  roots  different  from  1. 

Often  this  result  can  be  improved  further. 

Let  CO  and  ??  be  two  different  characteristic  roots  of  an  arbitrary  square  matrix  A,  and 

X={Xi,  X2,    .    .    .,  Xn) 

a  characteristic  vector  belonging  to  co  with  regard  to  the  rows  of  A  and  y  a  characteristic  vector 
belonging  to  rj  with  regard  to  the  columns  of  ^.  It  follows  from  E.  Schmidt's  theory  of  linear  in- 
tegral equations  that  JC  and  y  are  orthogonal.  This  can  easily  be  proved  independently.  We  only 
use  it  for  the  special  case  that  A  is  stochastic  and  aj=l. 

Since  JC=(1,  1,  .  .  .,  1)  is  a  characteristic  vector  belonging  to  co=l  with  regard  to  the  rows,  we 
have  to  prove  the  following  theorem. 

Theorem  8  [13].    Let  A  he  a  stochastic  matrix.    If  rj  is  a  nontrivial  characteristic  root  and 

y=(yi,  2/2,  •  •  .,  Vn) 

a  characteristic  vector  belonging  to  rj  with  regard  to  the  columns  of  A,  then 

2/1+2/2+    .   .   .   +2/«  =  0.  (18) 


\ 


Proof.     We  have 


^2/x=Z)  a«x2/.  (X=l,2,  ...,n).  (19) 
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If  we  add  these  equations,  then  we  obtain  by  (15) 

r](yi+y2+  .  .  .  +yn)=yi+y2+  .  .  ■  +?/»• 

This  proves  (18)  since  7j=l. 

If  a  matrix  A  is  reduced,  then  its  characteristic  determinant  is  the  product  of  characteristic  de- 
terminants of  unreduced  matrices  and  the  characteristic  roots  of  A  are  the  characteristic  roots  of  these 
unreduced  matrices.    Hence  it  is  sufficient  if  we  consider  unreduced  matrices. 

Theorem  9  [13].  Let  ^=(a«x)  be  an  unreduced  stochastic  matrix  and  hi,  hi,  .  .  .,  hn  arbitrary  numbers. 
^Denote  the  matrix  (a^\  —  h\)  by  B.     Then  B  has  the  trivial  characteristic  root 


o:'=l-JZK  (20) 

x=i 

and  the  other  roots  are  the  nontrivial  characteristic  roots  of  A  with  the  respective  multiplicities. 
Proof.     It  follows  from  (15)  and  (20)  that 

Z)  (a«x-/ix)=l-Z)  ^x=co'; 

X=l  X=l 

hence  co'  is  a  characteristic  root  of  B  and  (1,1,  .  .  .,1)  is  a  characteristic  vector  belonging  to  co'  with 
'regard  to  the  rows.    Since  A  is  unreduced,  co=l  is  a  simple  root. 

Let  7j  be  another  root  of  A  belonging  to  the  vector  y={yi,y2,  ■  ■  -,  ?/«)  with  regard  to  the  columns. 
It  follows  from  (19)  and  (18)  that 

I  n  n  n  n 

X)  (a-cx— /^x)yK=X)  ««x?y«— Ax  X)  2/«=X)  (i,xyK=r,y\- 

,  Hence  77  is  a  characteristic  root  of  B  belonging  to  y  with  regard  to  the  columns.    It  follows  in  the  same 
svay  that  each  nontrivial  characteristic  root  of  5  is  a  characteristic  root  of  A. 

Since  the  roots  of  the  characteristic  equation  change  continuously  if  the  elements  of  the  matrix 

'are  changed  continuously,  it  follows  that  the  corresponding  roots  of  ^1  and  B  have  the  same  multiplici- 

ities  unless  0/  is  a  characteristic  root  of  A.     In  this  case  the  multiplicity  of  co'  in  B  is  greater  by  one. 

It  follows  from  theorem  8  that  we  can  apply  all  the  theorems  on  bounds  for  the  characteristic 

roots  to  B  in  order  to  obtain  bounds  for  the  nontrivial  characteristic  roots  of  A.     This  method  is 


especially  efficient  if  for  each  m,  max  |a 
Let  us  consider,  for  instance 


A- 


\(lpm- 

dam] 

(p,  o-=l, 

2,  . 

.33 

.27 

.21 

.19^ 

.42 

.  14 

.23 

.21 

.44 

.26 

.  12 

.  18 

.42 

.27 

.22 

.09 

n;  p,  (T9^m)  is  relatively  small. 


'We  choose  Ai=.42,  hi^ .27 ,  A3  =.22,  ^4=19  and  obtain 

r-.09  0       -.01 


B- 


0 
.02 


0       -.13         .01 

02     -.01      -.10     -.01 

0  0  0       -.  10 
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The  trivial  characteristic  root  of  5  is  —.1;  the  other  characteristic  roots  are  the  roots  of  the  principal 
minor  ^44  of  order  3  of  B.  It  follows  from  theorems  1,  5,  and  6  applied  to  Bu  that  one  nontrivial  charac- 
teristic root  of  A  must  lie  in  the  interval  —  .14<0<  — .12  and  the  other  two  in  the  interior  of  at  least 
one  of  the  ovals  l2+.09||2+.10|^  .0003, |2-f.lO||2  +  .13|^  .0003. 

The  following  special  case  is  of  interest: 

Theorem  9.  Ij  h  is  one  of  the  elements  of  the  main  diagonal  of  a  stochastic  matrix  while  all  the  other 
elements  of  the  column  of  b  equal  c,  then  b-c  is  a  characteristic  root. 
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115.  Matrix  Inversion  and  Solution  of  Simultaneous  Linear 
Algebraic  Equations  With  the  IBM  604  Electronic 
Calculating  Punch 

George  W.  Petrie,  III  * 
Introduction 

Current  computational  literature  is  replete  with  many  descriptions  of  methods  for  matrix  inversion. 
(See  references.)     Gutshall  [1]  points  out  the  necessity  for  further  statistical  study  of  types  of  matrices 
which  may  be  subjected  to  numerical  inversion,  and  also  suggests  a  comparative  study  of  known  methods 
of  inversion.     Dantzig  [2]  demonstrates  in  his  simplex  method  a  recurring  need  for  obtaining  the  inverse 
of  certain  matrices  to  compute  optimum  programs.     In  fact  it  is  his  request  for  a  simple  fast  procedure 
■  for  inversion  of  large  order  Leontief  type  matrices  that  led  to  the  technique  presented  in  this  paper. 
;  The  procedm-e,  while  using  the  standard  elimination  method  [3,  4,  5,  6,  7,  8]  allows  of  simple  and  contin- 
uous processing  of  cards  through  a  reproducer,  the  604,  an  accounting  machine  (for  checking),  and  a 
I  second  reproducer.     This  cycle  is  repeated  A^  times  to  invert  an  A^th  order  matrix.     The  size  of  the 
matrix  is  not  limited  and  the  four  machines  are  operated  continuously  without  change  of  control  panels. 
Consider  the  matrix  equation,  AX=I,  as  describing  a  system  of  simultaneous  linear  equations  in 
the  x's.     By  simple  operations  (multiplying  equations  by  appropriate  constants  and  combining  with 
'  other  equations)  it  is  possible  (provided  A  is  nonsingular)  to  reduce  the  system  to  one  in  which  the 
coefficient  of  X  is  /.     In  this  case,  the  right  hand  member  of  the  matrix  equation  is  reduced  to  the  in- 
verse of  A.     Thus,  in  the  elimination  method,  one  starts  with  an  augmented  matrix  composed  of  the 
'  original  matrix.  A,  and  the  unit  matrix,  /.     As  the  original  matrix  is  reduced  to  the  unit  matrix,  the 
i  original  unit  matrix  is  simultaneously  reduced  to  the  inverse  of  the  original  given  matrix.     This  reduction 
may  be  carried  out  one  column  at  a  time.     After  (/<:— 1)  columns  have  been  reduced,  the  procedure  in- 
volves dividing  the  ki\\  row  by  a^j,  (to  obtain  1  in  the  A:th  column)  and  then  subtracting  the  product  of 
this  result  with  the  appropriate  constant,  a^^  from  the  ith  row  so  as  to  obtain  0  in  the  kth.  column  of  the 
ith  row.     This  latter  subtraction  is  carried  out  for  all  i9^k.     Columns  of  the  unit  matrix  numbered 
greater  than  k  are  unaffected.     The  ki\\  coliunn  is,  in  general,  completely  changed.     Thus  N^  elements 
enter  the  computation  at  any  one  cycle.     In  the  present  procedure,  the  ^th  column  (or  vector)  of  the 
left  hand  part  of  the  reduced  matrix  is  not  written  since  its  value  is  known  automatically.     Instead,  it 
!  is  replaced  by  the  kth  column  (or  vector)  of  the  right  hand  part  of  the  augmented  matrix  after  this  step 
'  in  the  reduction.     By  selecting  the  appropriate  formula  to  use  (as  listed  in  step  3  of  the  next    section) 
1  the  604  actually  brings  in  vectors  from  the  unit  matrix  as  needed.     The  same  machine  is  used  to  allow 
I  the  matrix  elements  to  be  reordered  after  each  step  in  the  elimination  so  that  the  new  pivotal  row  is  on 
top  and  the  new  pivotal  column  is  at  the  left.     Thus  intermediate  row  and  column  sorting  is  entirely 
eliminated.     It  is  through  the  elimination  of  all  collating  and  this  type  of  sorting  that  the  present  more 
rapid  machine  procedure  becomes  possible. 

The  accounting  machine  is  used  for  checking  purposes.  The  checking  operation  must  be  looked 
upon  as  occupying  one  step  in  the  continuous  flow  of  cards  through  the  four  machines.  Processing  would 
not  be  appreciably  accelerated  if  this  check  were  omitted  since  all  machines  are  running  simultaneously. 
It  is  important  to  point  out,  however,  that  the  checking  procedure  augments  the  original  A^^  cards  to 
{N^-\-N)  cards. 

Machine  Procedure 

The  original  A^'th  order  matrix  ((a,^))  is  punched  on  A^^^  cards,  one  element,  together  with  the  identify- 
ing i  and  j,  to  a  card.     An  additional  row  of  check  sums  is  appended  to  the  original  matrix.     These 
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values  are  defined  by  s^=  +  l-2a<^.     A  10-digit  fixed  decimal  system  is  used  (two  integers  and  eight 

i 

decimals).     Data  is  punched  as  follows: 

Columns:  Data 

1  to  3 Number  of  row  (i) 

4  to  6 Number  of  column  0) 

10  to  19 Value  of  element  (fty) 

20  to  29 Value  of  element  (a„) — first  card  only. 

In  all  cases  minus  signs  are  carried  as  a^-punches  over  the  right-hand  digit.  In  addition,  elements  of  the 
first  row  are  identified  with  x  and  y  punches  in  column  1  (denoted  xyl)  to  indicate  the  pivotal  row; 
elements  of  the  first  column  x  and  y7  punches  to  indicate  the  pivotal  column.  Elements  of  the  row  of 
s,  are  identified  by  x  and  ?/13  punches.  The  521  unit  is  used  to  supply  subsequent  x  and  y  punches  as 
needed. 

After  the  cards  are  prepared  and  sorted  to  row  within  column,  the  first  column  is  gang-punched  as 
listed  in  step  6.  After  this  preliminary  step,  the  following  six  steps  are  repeated  N  times  to  obtain  the 
inverse : 

1.  Reproduce  the  values  of  i  (columns  1  to  3)  and  o*^  (columns  10  to  19)  from  the  leading  (A^+1) 
cards  (with  identifying  xy7)  into  each  A^+1  of  the  remaining  (A^— 1)  (A^+1)  cards. 

Read  {xy7)  Punch  {nxy7) 

Digits  of  1  to  3 1  to  3 

10  to  19 20  to  29 

This  means  that  an  element  in  the  ith  row  of  the  leading  column  will  now  appear  on  the  same  card  with 
aU  other  elements  of  the  ith  row  of  the  matrix.  As  a  variation,  the  value  of  a^j  to  be  reproduced  may 
be  read  from  either  columns  10  to  19  of  xyl  cards  or  from  columns  20  to  29  of  nxy7  (no  xyl)  cards  which 
have  already  passed  through  the  punch  side  of  the  reproducer  for  this  step.  Thus,  in  practice,  the  oper- 
ator starts  with  the  N-{- 1  cards  bearing  the  designator  xyl  in  the  read  feed  and  all  others  either  in  the 
punch  feed  or  immediately  available  thereto.  After  reproducing  the  values  from  the  N-\-l  cards,  both 
stackers  are  emptied.  The  2(A^+1)  cards  are  placed  together  in  the  read  feed  and  4(iV+l)  cards  are 
next  obtained  from  the  two  stackers.  As  soon  as  a  sizeable  group  of  cards  are  generated,  one  may  take 
the  cards  from  the  read  stacker  to  the  next  operation  process  at  the  521-604  while  continuing  to 
reproduce  values  from  the  cards  that  came  from  the  punch  stacker  of  the  reproducer. 

In  both  cases  a  comparison  check  is  carried  on  all  reproduction.  In  addition,  the  reproduced  value 
of  i  (columns  1  to  3)  is  checked  for  double  punches.  The  reason  for  the  reproduction  of  i  is  apparent  at 
step  3. 

2.  Place  a  blank  card  having  a  different  distinguishing  color  and  xyl  punch  after  the  last  card  proc- 
essed in  step  1. 

3.  Calculate  values  of  bij  and  tj  according  to  the  following  formulas.  For  complete  generality, 
assume  that  (k—l)  steps  in  the  inversion  process  have  been  completed,  and  that  the  matrix  is  in  order  of 
row  within  column  as  given  in  the  following  array: 

dk.k  ...        (lk,j       •    .    •        dk.  N       (I'k.l        •    •    •        O'k. k- 1 


dij 


O'N.k 

Sk  .    .    .       Sj  ...       Sfc. 


%-l,*  •    •    •        Clk-l.k-1 
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alues  of  bij  and  tj=-\-\  —  Xl^o  a,ro  computed  by: 

xy — conditions 
^k.k=- —  pivotal  element  xyl,  xyl 

I  ^^•''^~n — ^^•'■'  ^^^'  *^^  pivotal  column  nxyl,  xyl 

hk,i=-\-   '^'^  for  ./VA:  pivotal  row  j^?/l,  n^?/7 

^A.  A: 

b{j=ai_j ~  •  at^k  for  i^k,J9^k  nxyl,  nxyl 

,  tk= 6't  check,  pivotal  column  na;?/l,  x?/ 7 

t  ^;  =  '^"; ''■''•■Sfc  for  ij^k  check,  other  nxyl,  nxyl 

0'k,k 

)ne  observes  that  the  computation  for  tj  is  identical  with  that  for  htj  where  i9^k.  The  values  com- 
uted  are  punched  directly  into  columns  30-39  on  all  nxyl  cards.  For  xyl  cards,  however,  the  values  are 
tored  and  punched  on  the  following  xyl  card.     This  results  in  the  following  array  of  values: 

i  bic,k  •     •     ■        Ok.j~\  •     .     .        Ok,N-l        Ok.N  •     •     .        Ok,k-2  Ok,k-l 

Ok  +  l,k       Ok-\-],k-H        ■    ■     ■       bk+i,j         .    .     .       Ok+i,N       bk-\.i,i       .    .     .       Ok-^-i,k-l 


bi,k  bij         .  .  .     bi^N        6i,i         .  .  .     bi_k-\ 


^N.k 

tk  .  .  .     tj  ...     tk-i 


\.k 


bk-i.k     bk-i,k+i     ■  ■  ■     bk-ij     .  ■  ■     bk-\,N     bk-\,\      .  .  .     bk-i.k-i 

'Notice  that  no  b  is  punched  on  the  first  card.  The  additional  card  of  step  2  was  added  to  obtain  the 
ast  value.  In  addition  to  the  above  calculations,  the  following  operations  arc  performed  simultaneously 
•n  the  521: 

(a)  Offset  gang  punch  digits  of  columns  4  to  6  into  7  to  9. 

(b)  Offset  gang  punch  xyl  into  a:?/4. 

(c)  Gang  punch  xyl  into  xyl  on  the  xijl  card  following  an  xyl  card. 

(d)  Emit  27/10  into  cards  following  the  xyl  which  follows  an  X7/1.     Stop  the  xylO  emission 

after  the  next  xyl  card, 
t  will  be  observed  that  these  operations  result  in  a  column  identification  of  the  last  card  (the  card  added 
it  step  2).  As  yet  there  is  no  row  identification,  but  this  will  be  supplied  by  step  1  of  the  following  cycle, 
f  needed  for  a  visual  check,  it  is  easy  to  remember  that  the  unidentified  row  is  one  gi-eater  than  the 
■olumn  number  that  is  already  on  the  card.  It  would  also  be  possible  to  prepunch  the  row  identification 
)efore  step  2.     Tliis  wouhl  involve  a  simple  consecutive  number  (h'ck. 
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4.  Tabulate  and  list  iV  lines  with  the  following  data: 

k     (read  from  first  card  of  column) 

i 

This  tabulation  is  used  to  check  the  accuracy  of  the  previous  steps.  The  last  value  should  differ  from 
+ 1  by  no  more  than  an  acceptable  rounding  error.  One  has  a  choice  of  checking  each  column  separately 
or  else  the  entire  matrix  as  a  whole. 

5.  Kemove  the  first  card,  of  which  no  further  use  is  made.     Take  the  next  N+1  cards,  and  place 
them  at  the  end  of  the  deck.     This  leaves  the  cards  in  the  following  array:  j 


^.v,ifc+i 

tk+l  •    •    •       tn 


br 


»+i 


"*-.*+i        •  •  •     bk,f/        bk,i        .  .  .     bk.k 

6.  Reproduce  into  a  new  set  of  cards  (or  tumbled  cards  already  half  utilized). 

Read  Punch 

Digits  of  1  to  3  1  to  3 

Digits  of  7  to  9  4  to  6 

30  to  39  10  to  19  on  all  cards 

a^lO  xy7 

3^2/13  xyU 

At  the  same  time  gang  punch  from  10  to  19  of  xyl,  xyl  into  20  to  29  on  all  nxyl,  xyl  cards. 

The  new  set  of  cards  will  be  in  such  order  that  the  pivotal  row  is  on  top,  the  pivotal  column  to  the 
left,  and  elements  of  both  are  marked  with  proper  identifying  x  and  y  punches.  Thus  these  cards  are 
ready  to  process  through  step  1  of  the  following  cycle.  After  N  cycles  the  values  obtained  at  step  5 
give  the  required  inverse , 

Solution  of  Equations 

To  solve  a  set  of  linear  simultaneous  algebraic  equations,  the  procedure  outlined  above  is  only  slightly 
modified.  The  same  control  panels  are  used  without  any  changes  or  additional  wiring.  The  original 
matrix  of  coefficients  is  augmented  by  the  vector  of  constants  and  row  of  negative  sums  of  column  ele- 
ments for  checking.  Thus  one  starts  with  a  matrix  of  {N-\-iy  elements.  After  step  1  on  each  cycle, 
the  first  (iV+ 1)  cards  {xyl)  are  discarded.     Thus  the  cards  which  pass  through  the  604  number  N{N+ 1), 

110 


(A^-1)  (A'+l),  (A^— 2)  (.V+l)  .  .  .,  1(.V+1).  At  step  4  the  check  value  is  approximately  —1,  —2, 
. .  .  ,  —  A^  on  the  successive  steps  and  should  differ  from  these  negative  integers  by  no  more  than  an 
acceptable  rounding  error.  At  step  5  the  first  card  is  removed.  No  other  rearranging  occurs.  Finally, 
one  obtains  the  solution  of  the  simultaneous  equations  as  a  vector  accompanied  by  a  check  sum.  This 
procedure  should  take  approximately  half  as  long  as  the  inversion  procedure. 

Sunitnary 

The  inversion  procedure  outlined  above  is  believed  to  be  faster  and  easier  to  perform  than  other 
methods  now  in  common  practice.  Further  inquiry  should  be  made  of  the  applicability  of  this  pro- 
cedure to  a  floating  decimal  calculation,  also  to  matrices  involving  complex  numbers.  The  author 
understands  that  members  of  Dr.  Dantzig's  group  are  investigating  the  problem  of  which  order  of  ele- 
ments allows  of  the  determination  of  the  most  acciu-ate  inverse.  Certainly  more  work  is  necessary  in 
this  area. 

For  large-order  matrices  this  procedm-e  may  be  used  to  continuously  process  cards  from  one  machine 
to  another,  keeping  all  machines  in  operation  simultaneously.  For  small-order  matrices,  the  number 
of  cards  is  insufficient  to  keep  all  machines  running.  In  this  case,  all  cards  would  be  processed  at  one 
machine  at  a  time.     Several  matrices  coidd  be  inverted  at  the  same  time. 

It  is  possible  to  perform  steps  1  and  6  on  a  single  type  519  document  originating  machine  with 
only  one  control  panel  if  a  control  punch  has  been  emitted  onto  the  cards  at  step  3.  This  modification 
would  be  useful  for  handling  low-order  matrices. 

If  desired,  it  is  also  possible  to  handle  a  row  check  similar  to  the  column  check.  In  this  case,  the 
check  sum  is  defined  by: 

5,=  — 1— X)a« 
J 

The  writer  expresses  his  acknowledgments  to  Dr.  George  B.  Dantzig,  U.  S.  Au'  Force  Comptroller's 
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the  development  of  a  fast  method  of  inversion  of  Leontief  type  matrices.  He  also  suggested  the  tech- 
nique whereby  the  unit  matrix  is  generated  vector  by  vector  as  needed,  resulting  in  the  equations  for 
bij  in  step  3.  Dr.  Sherman  developed  the  reproduction  used  in  step  1  in  a  paper  concerning  matrix 
inversion  with  the  602A.[9]  The  assistance  of  Miss  Cooper  was  realized  in  aiding  with  the  program- 
ming and  executing  the  entire  plan  into  successful  operation. 
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16.  Experiments  on  the  Inversion  of  a  16X16  Matrix' 

I  John  Todd  * 

Introduction 

Some  experiments  have  been  carried  out  in  the  Computation  Laboratory  of  the  National  Bureau 
of  Standards  on  the  inversion  of  a  certain  16X16  matrix,  using  the  following  three  methods:  (1)  G.  W. 
Petrie's  arrangement  of  the  Gauss  elimination  process,^  (2)  a  Monte  Carlo  process,^  and  (3)  an  iteration 
method.^     It  is  the  purpose  of  this  note  to  describe  and  compare  the  results  obtained. 

The  matrix  was  a  16X16  matrix  of  the  so-called  Leontief  type  [1,  2],*  representing  certain  inter- 
industry relations.  A  matrix  of  the  same  type,  but  of  order  40X40,  has  been  investigated  by  J.  L. 
Holley  of  the  Air  Comptroller's  Office,  USAF;  in  particular,  it  has  been  inverted,  using  the  UNIVAC. 

The  actual  matrix  inverted  was  B==I—A,  where  lOM  is: 

0  36265  0  0  0  0  0  0  4646  3169  8939  0  421  0  1355  0 

5158  0  0  0  0  0  0  0  1382  103  2466  0  120  0  6995  5 

188  0  0  9779  12795  2679  0  0  0  1062  0  0  0  909  0  5 

20  38  77  0  30  128  145  264  29  85  99  335  0  0  5  0 

69  2225  424  10425  0  128  0  275  882  874  83  0  601  2434  253  104 

49  173  4165  2398  3252  0  145  550  2705  86  17  0  3065  0  0  30 

128  913  1080  1209  696  319  0  66  3528  754  33  561  300  174  200  10 

3794  1109  12225  1835  2087  3890  8475  0  5322  2861  1986  2578  2464  8653  2822  277 

I   2915  905  1388  709  1361  638  1065  286  0  1867  5314  3475  2404  201  0  173 

I    869  1682  77  709  907  383  1937  22  1764  0  761  448  1563  134  1194  6747 

524  324  0  1501  121  64  97  0  382  1388  0  7960  1202  0  146  119 

326  23  0  3878  242  0  0  11  29  34  463  0  60  348  58  139 

j    10  0  0  250  136  0  0  0  0  137  1639  0  0  321  331  1288 

1   7845  2942  10297  2252  1996  4719  14286  24210  6616  6630  844  3475  421  0  19  0 

3537  2527  926  1877  1165  510  1840  385  3323  2895  3145  897  2764  642  0  515 

5444  2821  501  1772  1119  191  630  429  5410  1079  1374  4596  962  976  .5294  0 

Method  1 

The  actual  running  time  was  some  8  hours  on  the  604  Calculating  Punch.  The  error  in  the  check 
sums  carried  was  about  160  units  in  the  last  (eighth)  place.  The  resultant  matrix  is  denoted  by  G 
and  is  available. 


•National  Bureau  of  Standards. 

'  This  work  has  been  supported  by  the  Air  Comptroller's  OfTice,  USAF. 

'  See  the  previous  paper.    The  IBM  operations  were  carried  out  under  the  direction  of  Helen  V.  Ilaininar  of  the  XllS  Computation  Laboratory. 

>  These  were  earried  out  on  SEAC  under  the  direction  of  Karl  Ooldherp  of  the  XBS  Computation  Laboratory. 

*  See  also  [3]  for  description  of  the  inversion  of  a  38X38  matrix  of  this  tyiJe,  by  a  Gaussian  process,  on  the  Aiken  Relay  Computer,  Mk.  IL 

li:j 


Method  2 

The  method  described  by  G.  E.  Forsythe  and  R.  A.  Leibler  [4]  was  used  m  the  Monte  Carlo  experi- 
ments.  The  "random"  numbers  used  were  generated  by  the  process  suggested  by  Olga  Taussky: 
residues  modulus  2^  of  powers  of  an  odd  power  of  5.  These  residues  have  period  2^  and  are  generated 
by  a  single  operation  ("low"  multiplication)  on  SEAC. 

The  first  experiment  involved  carrying  out  1,000  walks  per  row;  this  took  some  21  minutes.  The 
second  used  10,000  walks  per  row  and  took  3  hours  15  minutes. 

Reading  in  the  (matrix  and  the)  instructions  took  about  21  minutes ;  printing  out  the  matrix  took 
about  11  minutes.  The  complete  matrices,  denoted  by  Mo  and  M,  were  printed  out  and  are  available. 

Method  3 

The  iteration  method  used  was  the  standard  one  [5,  6], 

Z„+,  =  Z„(2-BZ„),         n>0. 

The  initial  approximation  actually  used  was  X^=M^,  that  is,  that  obtained  after  the  1,000-walk- 
Monte  Carlo  experiment,  but  not  very  different  results  would  have  been  obtained  in  the  present  case 
if  we  had  taken  X^=I.  Each  iteration  takes  about  a  minute  on  SEAC.  Printing  out  takes  about  11 
minutes  as  before,  and  about  15  minutes  would  be  required  for  reading  in  the  matrix  and  the  instructions. 

The  following  material  is  available:  The  original  approximation  X^=M^  and  the  following  iterates 
Xj,  {=1(1)10,15,20.  In  addition,  for  checks,  the  matrices,  7?i5=BXi5— 7  and  7?2o=-BX2o— /,  were 
printed  out. 

The  results  have  not  yet  been  analyzed  fully,  but  some  idea  of  their  accuracy  can  be  obtained  from 
the  following  table: 


X.=Mo 

Element  (1,1) 

Element  (7,11) 

1.  03528  3690 

.00348  1200 

Mx 

1.  02289  5670 

.  00301  7210 

X, 

1.  02259  5947 

.  00322  1541 

A'^io 

1.  02347  2122 

.  00320  9509 

X,5 

1.  02349  5929 

.  00320  8423 

X20 

1.  02349  6765 

.  00320  8375 

G 

1.  02349  680 

.  00320  835 

Rii*  R20*         {G-X20)** 

Maximum  element 117538  4726  387 

RMS  element. 1137  44  25 

(1,1)--- -9703  -382  4 

(7,11) -568  23  3 

*In  units  of  ninth  decimal. 

**In  units  of  eighth  decimal  (the  elements  of  Xio  being  rounded) 

Remarks 

In  the  present  case  it  would  probably  have  been  more  efficient  to  omit  the  Monte  Carlo  calculation 
and  start  off  with  Xo=I. 

Using  current  figures  of  cost  there  is  little  to  choose  between  the  estimates  for  the  IBM  method 
and  the  Monte  Carlo — iteration  method.  However,  if  the  Monte  Carlo  stage  was  omitted,  and  if 
faster  input  and  output  devices  were  available***,  there  is  a  considerable  margin  in  favor  of  the  SEAC 
operation. 
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The  SEAC  operation  would  handle  larger  matrices,  say  up  to  order  27,  in  the  combined  high  speed 
memories.     If  still  higher  order  matrices  were  to  be  handled  use  could  be  made  of  the  magnetic  tape.*** 

Further  experiments  on  the  inversion  of  the  same  matrix  B  and  on  the  determination  of  its  charac- 
.teristic  values  are  planned. 

***(Added  September  1952)  Such  devices  are  now  in  operation  and  current  rates  of  input  and  output  to  magnetic  wire  are  about  50  words  per  second.  Using 
imagnetic  tapes  for  intermediate  storage,  operations  on  matrices  of  order  204  have  been  successfully  caiTied  out;  the  total  capacity  presently  available  is  about 
80,000  words. 
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17.  A  Method  of  Computing  Eigenvalues  and  Eigenvectors 
Suggested  by  Classical  Results  on  Symmetric  Matrices 


Wallace  Givens 


17.1.  Introduction 

The  problem  posed  is  precisely  that  considered  by  Goldstine  in  his  lecture  before  the  Symposium: 
to  devise  a  method  of  calculating  all  the  eigenvalues  and  eigenvectors  of  a  real  s\Tnmetric  matrix  suitable 
for  use  with,  an  automatic  sequenced  high-speed  digital  computer.  It  can  be  regarded  as  a  source  of 
satisfaction  that  the  basic  technical  device  proposed  here,  that  of  a  sequence  of  rotations  in  coordinate 
planes,  is  the  same  as  that  used  by  von  Neumann  and  Goldstine.  An  essential  difference  is  that  the 
plane  rotations  are  used  only  to  reduce  the  data  of  the  problem  from  |n(/i+l)  numbers  to  2;i  — 1,  after 
which  the  roots  of  the  matrix  are  obtained  by  operations  with  a  Sturm  sequence  of  poljTiomials.  Thus 
for  a  matrix  of  order  77  =  100,  storage  for  5,050  numbers  is  initially  required,  but  in  the  first  stages  of 
the  problem  this  data  can  be  effectively  processed  in  segments  until  only  199  numbers  need  be  used 
when  the  eigenvalues  are  calculated.  If  the  10*  components  of  the  n  eigenvectors  are  required,  storage 
requirements  must  again  grow  large,  but  the  interplay  between  the  internal  and  external  memories 
appears  favorable.  The  advantages  as  to  memory  requirements  are  like  those  of  the  von  Xeumann- 
Goldstine  method  and  differ  from  techniques  that  require  the  calculation  of  a  sequence  x,  Ax,  A^x,  .  .  ., 
of  vectors  and  that  involve  operation  with  the  whole  matrix  A  at  each  step. 

The  fundamental  problem  of  the  effect  of  round-off  error  is  not  considered  here.  It  seems  likely 
that  the  conclusions  stated  by  Goldstine  will  be  applicable,  but  this  needs  fm-ther  study. 

Added  in  proof  September  10,  1952:  A  detailed  error  analysis  has  now  been  made,  and  the  stability-  of 
the  proposed  method  of  finding  eigenvalues  can  be  fully  guaranteed. 

17.2.  Summary  of  Results 

By  a  sequence  of  at  most  ^(n  —  l)(n  — 2)  full}'  determined  rotations  in  coordinate  planes,  a  real 
symmetric  matrix  A  of  order  n  can  be  reduced  to  the  form 

0 
0 
0 
0 


s=  < 


"1 

^1 

0 

0 

0 

^1 

"2 

^2 

0 

0 

0 

/32 

"3 

^3 

0 

0 

0 

)33 

"4 

^ 

^n-2  0 

/3„_2  ttn-l  0n-\ 

0  /3„_i  a„ 


(1) 


'  Oak  Ridge  National  Laboratory  and  the  University  of  Tennessee. 
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The  successive  principal  minors  fi{\)  of  \I—S  and  the  characteristic  equation  of  S',/„(X)  =  0,  (which 
is  the  same  as  that  of  A)  are  obtained  by  the  recursion  formula 

/,(X)  =  (X-«,)/,_i(X)-(/3,_,)y,_2(X),  (2' 

where/_i  =  0,/o=l,  and  ?  =  1,  2,   .   .  .,  n. 

If  any  ^j=0,  S  decomposes  into  the  direct  sum  of  two  symmetric  matrices,  and  the  determination 
of  eigenvalues  and  eigenvectors  of  S  (and  hence  of  A)  can  be  replaced  by  the  same  problems  for  the  two 
smaller  matrices.  For  simplicity  of  statement,  we  assume  /3j?^0  for^^l,  .  .  .,  n— 1,  although  'small" 
but  nonzero,  /3y  should  be  dealt  with  more  fully  in  setting  up  details  of  a  computational  procedm-e. 
Then/„(X),/„_i(X),  .  .  .,/i(X),/o=l  form  a  Sturm  sequence  of  functions.  Hence  the  number  of  roots 
of /„(X)  greater  than  any  chosen  real  number  a,  with /„ (a)  5^0,  is  equal  to  the  number  of  variations  in 
sign  of  the  sequence /„  (a), /„_i  (a),  .  .  ., /i(a), /o(a)  =  l-  Moreover,  the  roots  of  each/,-  are  separated 
by  those  of /i_i  (i=2,  .  .  .,  n).  The  coefficients  of/„(X)  can  be  computed  if  desu-ed,  but  the  recursion 
formula  yields /„  (a)  for  any  a  without  much  more  computation  than  would  be  required  if  the  coefficients 
were  known.  The  knowledge  of  a  Sturm  sequence  (without  additional  computation)  makes  it  appear 
plausible  that  the  roots  of /„(X)  =  0  can  now  be  computed  to  any  required  accuracy  with  as  much  ease 
as  should  be  expected  from  the  inherent  difficulty  of  the  problem. 

Having  found  an  eigenvalue  p,  an  eigenvector  x  of  S  can  be  found  by  solving  the  equations 
(pI—S)x=0  for  the  ratios  of  the  Xt]  this  is  trivially  done  by  the  recursion  formula 

x,+i=(i3i)-'[(p— o!i)a;j— /Sf-iXi-i], 

where  i=l,  .  .  .,  n—1,  j3q=Xo=0  and  for  i=n,  /3„=1,  and  Xn+i  =  0  is  a  check  on  the  computation.  To 
improve  the  accuracy,  an  approximation  to  p  could  be  used  to  find  an  approximate  eigenvector  and  then 
both  X  and  p  simultaneously  refined  by  any  desired  iterative  process.  Since  only  2n—l  numbers  need 
to  be  stored  to  specify  S,  the  chosen  iterative  process  can  be  carried  out  for  matrices  of  large  order, 
using  only  the  internal  memory  of  a  high-speed  computing  machine.  To  find  the  corresponding  eigen- 
vector of  A  from  one  of  S,  it  is  only  necessary  to  "record"  the  reduction  of  A  to  S:  if  S=T' AT  with 
T=T2zT2i  .  .  .  Tn-\n  (cach  Tpq  being  a  plane  rotation),  so  that  T  is  obtained  if  the  columns  of  the 
unit  matrix  are  operated  on  precisely  as  are  the  columns  of  A,  and  Sx  =  px,  then  Ay  =  py  with  y=  Tx. 

17.3.  Heuristic  Considerations 

In  a  problem  of  the  nature  here  considered,  it  seems  certain  that  a  Avide  variety  of  methods  of  solu- 
tion will  be  needed  because  of  varying  requirements  for  accuracy,  size  of  matrix  involved,  use  of  the 
method  as  a  subroutine,  etc.  In  particular,  the  fact  that  the  largest  eigenvalue  of  A  is  the  maximum  of 
x'Ax  for  x'x=l  is  basic  in  many  proposed  methods  of  finding  the  eigenvalues.  From  the  standpoint  of 
the  algebraist,  this  use  of  the  order  properties  of  the  real  field  to  find  (a  rational  approximation  to)  the 
roots  of  a  polynomial  with  necessarily  rational  coefficients  is  suspect. 

Against  this  it  may  be  argued  that  to  disregard  the  matric  origin  of  the  roots  (for  example,  by  direct 
computation  of  the  coefficients  of  the  characteristic  equation  in  terms  of  determinants)  is  known  to  be 
hopelessly  inefficient.  Moreover,  one  wants  the  roots  of  the  characteristic  equation,  and  this  is  quite 
definitely  a  "nonrational"  problem  for  which  iterative  methods  of  finding  successive  approximations 
are  highly  appropriate. 

We  are  therefore  led  to  seek  a  compromise  in  which  "nearh'  rational"  methods  are  used  to  reduce 
the  complexity  of  the  data  presented  to  the  computer  (specffically,  from  ^n(n-\-l)  numbers  to  2n—l) 
and  then  to  accept  the  "irrationality"  of  the  problem  by  solving  for  the  roots  and  eigenvectors  by 
iteration. 

A  purely  mathematical  result  is  now  of  interest^:  Every  symmetric  matrix  of  rank  r  with  elements 
in  a  principal  ideal  ring  R  is  congruent  with  a  matrix  of  the  form  (1),  see  above,  w^here  af=/3,_i  =  0  if 
i^r.  One  notes  that  there  is  here,  as  elsewhere  in  this  memorandum,  no  requirement  that  the  matrix 
be  definite.     The  congruence  referred  to  is  not  restricted  by  a  requirement  of  orthogonality,  but  the 

'  C.  C.  MacDuffee,  Theory  of  matrices,  p.  54  (Berlin,  1933). 
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reduced  form  (1)  is  highly  rational  (note  the  requirement  on  R)  and  is  so  easy  to  obtain  that  AlacDuffee 
refers  to  a  similar  result  for  skewsymmetric  matrices  (given  by  Kronecker  in  1883)  to  indicate  a  proof 
of  the  theorem.  It  is  a  little  surprising,  but  definitely  of  interest,  that  requiring  the  transforming  matrix 
(Tin  T'AT=S)  to  be  orthogonal  (a)  does  not  prevent  the  existence  of  T  (indeed,  the  diagonal  form  is  a 
special  case  of  (1)),  (b)  introduces  no  irrationality  beyond  a  sequence  of  square  roots,  and  (c)  permits 
the  step-by-step  computation  of  T  as  a  product  of  rotations  in  the  coordinate  planes.  Factoring  T 
into  a  product  of  more  elementary  transformations  is  quite  analogous  to  the  factorization  of  P  and  Q 
into  elementary  matrices,  which  is  the  key  to  the  equivalence  reduction  M^PMQ  of  a  general  matrix 
M  used  in  the  successive  elimination  method  of  solving  a  system  of  linear  equations. 

17.4.  Reductioa  to  Triple  Diagonal  Form 

The  reduced  form  given  in  (1)  can  certainly  be  obtained  when  n=l  or  2,  establishing  satisfactorily 
the  basis  for  the  following  induction.  Nevertheless,  it  may  be  instructive  to  consider  the  geometry  of 
the  case  n=3.  To  get  the  1,  3  (and  3,  1)  element  of  a  symmetric  matrix  to  be  zero,  we  require  that  the 
(orthogonal)  unit  basis  vectors /i  and/3  be  conjugate: /lS/3=Si3=0.  Starting  from  an  orthogonal  basis 
«i,  «2,  ez  (e'iei=8ij),  it  is  enough  to  take/i=ei  and /a  in  the  intersection  of  the  planes  e'lAx^O  and  <?ix=0, 
that  is, /a  is  in  both  the  plane  conjugate  to  e^  and  the  plane  orthogonal  to  ei.  For  these  planes  to  coincide, 
e'iAe2=e[Ae3=0  or  ai2=Oi3=0,  and  ei  is  already  an  eigenvector  with  value  an.  In  the  general  case,  an 
inductive  step  would  be  rendered  unnecessary  if  it  should  happen  that  the  r-dimensional  space  spanned 
by  the  first  r  basis  vectors  chosen  was  conjugate  to  its  (n—r) -dimensional  orthogonal  space. 

To  establish  the  induction,  let  <"=aj?^  i,j=l,  •  •  .,nhe  the  elements  of  A^'\  and  suppose  a\l^  is 
the  first  nonzero  element  in  the  sequence  a[\\  a['^,  .  .  .,  a"„\  Interchanging  the  2d  and  ^th  columns  and 
then  the  2d  and  kth.  rows,  produces  a  new  symmetric  matrix  A^^^  with  afzVO.  One  could  also  require 
a^^2  =max  {a[\\  a[^^ ,  .  .  .,  a^^}  ii  this  should  prove  desirable  to  improve  error  estimates.  Also  there  is 
no  absolute  need  to  actually  change  the  position  of  the  components  as  recorded  in  the  machine,  the 
specification  of  "next  number  in  the  column  (row)"  in  the  order  code  needs  only  to  be  adjusted. 
Also,  if  aiy  =  0,  one  later  rotation  that  would  be  required  in  the  general  case  will  be  unnecessary,  so 
there  is  no  need  to  "count"  this  operation. 

If  we  now  set  A''' =  TLA''' T,,,  where 


23> 


0 

—s 
c 


ln-3 


(3) 


I   we  get  the  following  matrix,  where,  for  convenience,  we  have  omitted  the  superscript  (2)  on  the  elements 
:  oiA^'K 


^(3): 


an  cai2  +  sai3  can  —  sau  an 

Cai2  +  Sai3  C(ca22  +  S023)+S(ca23  +  S033)  C(ca23  — Sa22)+S(ca33  — 8023)  Ca24  +  S034 

can  —  sau  €(0023  +  8033)— 8(0022  +  8023)  0(0033  — 8023)  — 8(0023  — SO22)  cosi  — so24 
ou                            co2^  +  sa3^                                    co3i  — sa2i  a** 


Oi„ 


C02n  +  S03n 


C03n  —  Sa2„ 


.        C02n  +  Sa3n 

.     ca3„  —  sa2n 
0^„ 


IV.) 


Rule:  To  form  B^T-iATa,  make  the  following  replacements,  where  (  ),•  is  the  ith  column  and  (  )'  is  the 
tth  row:  U),-^c(yl),  +  sU),-;  U),->c(^),-sU),;  (^T.,)W->cUT„)'  +  sUT,,)';  UT,y)'-^c(^T,,)'-s(^T,,)<.  Use 
c=  [l  +  o,-i,,(ai-i,i)-M~^  and  s  =  ca,_i,,(a,_i,,)-*. 


Taking 


c=- 


<Jl2 


and     s  = 


^13 


V(<^12)^  +  (Ctl3)^ 


or 


we  have 


c=[l+F]-^,     s=c/:     and     Ar=ai3(oi2)"S 


a<i'=0. 


(4) 

(5) 
(6) 


Evidently  a  rotation  in  the  Xi—x^  plane  will  make  ai\!=0  and  retain  the  condition  a/|'=0  (since  Xi 
and  3-3  are  unaffected).     After  at  most  n— 2  such  rotations,  /l'"^  will  have  the  form 


^(«)  = 


0     0 


5 


(7) 


where  B  is  symmetric  of  order  n— 1.     We  now  make  the  inductive  hypothesis  that  an  orthogonal  con- 1 
gruence  affecting  only  the  rows  and  columns  numbered  3,  4,  .  .  .,  n,  will  carry  the  matrix  B  to  the 
reduced  form  (1).     These  do  not  disturb  the  zeros  in  the  first  row  or  column  of  A^'^K     (Note  that  the 
second  row  and  column  of  yl^"\  which  contain  the  first  of  B,  are  not  "rotated".)     We  therefore  have 

Theorem  1.  A  well-deiined  sequence  of  orthogonal  congruences,  affecting  at  each  step  only  two  rows 
and  columns,  will  carry  an  arbitrary  real  symmetric  matrix  into  a  matrix  S  such  that  S(>==0  if  \i—j\'^l. 

17.5  The  Characteristic  Equation 

Forming  'Kl—S  for  the  reduced  form  (1)  oi  S  and  expanding  its  determinant  in  terms  of  the  last 
row  and  column,  we  get 

/,(X)  =  (X-a,)/i-i(X)-(/3:-i)y.-2(X),  (8) 

where /i(X)  is  the  determinant  of  the  minor  of  order  i  in  the  upper-left  corner  of  \I—S.     If  we  set 

/_i  =  0     and    /o=l,  (9) 

the  formula  is  valid  for  i=l  and  2. 

If  /3i_i  =  0,  each  of  the  i—l  real  roots  of/i_i(X)=0  is  also  a  root  of/i(X)=0  and  hence  of  all/^(X)=0 
for  j=i+l,  .  .  .,  n.  As  noted  in  section  17.2,  /3i=0  causes  S  to  be  the  direct  sum  of  a  matrix  of  order 
i  and  one  of  order  n—i,  and  so  the  problem  reduces  to  two  problems  involving  matrices  of  lower  order. 
We  therefore  assume 

)S,?^0     fo^^=l,2,  .  .  .,  n-1.  (10) 
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Under  this  assumption,  for  any  chosen  vahie  of  X,  no  two  consecutive/i(X)  can  equal  zero,  since  this 
would  imply /o=0.  Hence  \I—S  is  regularly  arranged  and  the  classical  result  of  Darboux  ^  on  the  signa- 
ture of  a  quadratic  form  can  be  applied.     This  gives  the  second  sentence  of  the  following  theorem. 

Theorem  2.  The  functions  fn,  f„-i,  .  .  .,  /o=  1  defined  by  the  recursion  formula  (2)  are  a  Sturm  sequence 
if  ^i9^0  for  i=l,  2,  .  .  .,  n—l.  The  number  of  eigenvalues  of  S  greater  than  a,  provided  fn{a)  9^ Q,  is  the 
'■■  number  of  variations  of  sign  in  the  sequence  fja),  f„-i(a),  .  .  .,  fi{a),\.  The  roots  of  each  ft  (\)  =  0  are 
distinct  and  are  separated  by  those  o//i_i(X)=0. 

To  complete  the  proof  of  the  theorem,  we  use  induction  on  the  following  assertion,  the  case  i='S 
being  easily  established: 

the  1!  — 2  roots  of /i_2(X)=0  are  distinct  and  properly  separate  the  i—l  distinct  roots  of/i_i(X)=0.     (11) 

;  Explicitly,  no  root  of/j_2(X)  =  0  is  also  a  root  of /i_i(X)  =  0.    Let  the  roots  of/<_2(X)  =  0  be  o-i<o-2<  .  .  .  <C<^i-2, 
'  and  those  of/(_i(X)  =  0  be  pi<p2<   .  .  .  <Pi_i,  and  we  have  assumed 

Pi<(^i<p2<o-2<    .    .    .    <o-j;-i<Pj<o-:/<Pi+i<    •    •    •    <P<-1-  (12) 

Writing  the  recursion  formula  in  the  form 

y,(X)  =  fX-a,)(X-p,)(X-p2)    .   .   .   (X-p,_,)-(^0'(X-<^i)(X-^2)    .   .   .   (X-(r,_2),  (13) 

we  find  on  substituting  pj  for  X  that  the  signs  of  fi(pj)  are  as  indicated: 

/,(x):    (-1)^  (-1)'-'        (-1)^-^       (-1)^-^    (-1)*-^       {-iy-^+' -1         +1      ,^  .. 

> 1 1 1 1 1 1 1 1 1 1 -— (14) 

—   00  CTi  0-2  O-j  •    •    •  <^i-2  +   °° 

Pi  P2  P3        •    •   •  P)  PJ+I  P<-1 

Since/i(X)  =0  has  exactly  i  real  roots,  they  are  distinct  and  are  properly  separated  by  the  roots  of/i_i  =  0 
All  necessary  properties  that  the  sequence  /„(X),/„_i(X),.  .  .,  /o=l  be  a  Sturm  sequence  *  are  now 
evident. 

17.6.   Number  of  Multiplications  Involved 

The  elements  in  the  second  and  thu-d  columns  of  the  matrix  A^^^  displayed  in  section  17.4  can  be 
calculated  w4th  4n+8  multiplications.  Since  the  (2,3)  and  (3,2)  elements  are  equal  and  the  sum  of  the 
(2,2)  and  (3,3)  elements  is  unchanged  by  the  rotation  (so  that  the  (3,3)  element  can  be  computed  by  addi- 
tion and  subtraction),  this  can  be  reduced  to  4n+4.  Since  n—2  of  these  rotations  may  be  required  at 
this  step  of  the  induction,  we  may  need 

X;  (k-2){ik  +  4)=^{n'-7n+6) 

k=n  "J 

multiplications  to  reduce  the  original  matrix  to  the  form  (1).  The  computation  of  ^(n-l)(n-2)  values 
of  c  and  s  are  also  required.  This  could  be  done  with  (n-l)  (n-2)  divisions,  the  same  number  of  multi- 
plications and  half  as  many  square  roots.  Neglecting  powers  of  n  below  the  third,  we  conclude  that 
approximately  ^n'^  multiplications  are  requu-ed.  Thus  at  a  cost  of  about  one  and  one-third  matrix 
multiplications,  the  reduction  to  the  triple  diagonal  form  (1)  can  be  accomplished. 

Equations  (2)  involve  only  the  squares  of  the  i3,,  and  we  suppose  these  have  been  computed  and 
recorded.  Then  a  single  computation  of  the  Sturm  sequence  requires  2(n-l)  multiplications.  If  the 
eigenvalues  are  known  to  lie  between  -1  and  +1,  the  largest  eigenvalue  can  be  found  to  s  binary  place 
accuracy  (between  p  and  p+2-')  by  2(n-l)(s+l)  multiplications.  Using  the  method  in  its  crudest 
form,  one  would  require  only  2(s+l)(ri-l)n  multiplications  to  get  all  n  eigenvalues  to  this  accuracy. 

'  See  p.  57-58  of  reference  Riven  in  footnote  2. 

♦  I  am  indebted  to  my  colleague  Walter  Snyder  for  suggesting  that  these  functions  probably  formed  a  Sturm  sequence. 
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Because  this  is  of  the  order  of  magnitude  of  a  constant  multiple  of  ti^,  and  hence  comparable  with  terms 
previously  neglected  when  rOs,  we  do  not  attempt  a  more  careful  estimate.  For  smaller  values  of 
n,  say  n=s=40,  the  crude  determination  of  the  eigenvalues  could  involve  2n^  multiplications  or  two 
matrix  multiplications. 

17.7.  Relation  to  Other  Methods 

An  inspection  of  other  methods  proposed  for  the  eigenvalue  problem  will  show  that  in  a  number 
of  cases  the  triple  diagonal  form  and  the  recursion  formulas  for  the  principal  minors  both  occur.®  Here 
the  fundamental  paper  of  C.  Lanczos  deserves  mention.®  ^  (This  also  contains  much  material  relating 
the  question  to  differential  and  integral  as  well  as  algebraic  problems.)  The  same  two  basic  devices 
are  also  to  be  found  in  a  recently  published  paper  by  W.  Karush  ^  and  in  the  work  of  Hestenes  and 
Stiefel,  as  presented  elsewhere  in  this  volume. 

« Indeed  for  operators  in  Hilbert  space  the  triple  diagonal  form  is  well  known  under  the  name  "Jacobi  Form." 

'  C.  Lanczos,  An  iteration  method  for  the  solution  of  the  eigenvalue  problem  ot  linear  differential  and  integral  operators,  J.  Research  NBS  45,  255-282 
(1950)  RP  2133. 

'  Entirely  through  oversight,  this  reference  was  omitted  when  the  material  was  presented  to  the  Symposium.  This  was  in  spite  of  the  fact  that,  although 
I  had  not  seen  the  paper  previously,  it  had  been  called  <o  my  attention  during  the  conference.    Dr.  W.  Karush  kindly  rectified  my  error  in  the  discussion. 

'  W.  Karush,  An  iterative  method  for  finding  characteristic  vectors  of  a  symmetric  matrix,  Pacific  J.  Math.  1,  233-248  (19511. 
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18.  Computations  Relating  to  Inverse  Matrices 

Jack  Sherman^ 

Introduction 

In  many  practical  problems  it  is  frequently  desired  to  obtain  inverses  of  matrices  that  differ 
from  one  another  in  some  systematic  manner,  for  example,  in  the  elements  of  a  particular  row  or  column. 
;It  would  obviously  be  advantageous  to  develop  computational  methods  for  obtaining  these  inverses 
directly  from  one  another,  rather  than  from  the  original  matrix.  Some  pertinent  results  are  described 
below. 

Adjustment  of  an  Inverse  Matrix  Corresponding  to  a  Change  in  One  Element 
I  of  a  Given  Matrix  ^ 

j  A  simple  relationship  has  been  derived  by  means  of  which  the  inverse  of  a  given  matrLx 
can  be  "adjusted,"  as  a  result  of  changing  one  element  in  the  original  matrix.  The  set  of  equations 
may  be  \vritten  as 

Brj=brj-.,r\^     >         r  =  l,2,...,n,    j  =  l,2,...,n, 

provided  that  l-^-bsR^ciRs^O. 

In  these  equations  Brj  denote  the  elements  of  (B),  the  inverse  of  (A);  brj  denote  the  elements  of 
(6),  the  inverse  of  (a).  ^Urs  denotes  the  change  in  the  element  Grs,  that  is  Ars—C'Rs-\- ^clrs-  It  is  seen 
from  the  preceding  equations  that  if  AaRs=  —  llbsR,  then  (^1)  becomes  singular. 

Adjustment  of  an  Inverse  Matrix  Corresponding  to  Changes  in  the  Elements 

of  a  Given  Column  ^ 

If  (A)  differs  from  (a)  in  the  elements  of  the  kth  column,  then  the  equations  for  computing  the 
elements  of  (B)  from  (b)  are  as  follows: 

,N,    j=l,2,  .  .  .,N, 
Bk)=bkj/Zk,  i=l,2,  .  .  .,A^, 

in  which 


Bij — bij — ZiBkj, 
Bkj=bkj/Zk, 

'^=1,2,   . 

.  .,k-l,k+l, 
.  .,N, 

N 

Zi=^birArk, 
r  =  l 

r=l,2,  . 

.  .,N. 

The  foregoing  equations  can  be  conveniently  utilized  to  carry  out  the  numerical  computations  on  IBM 
machines  for  obtaining  (B)  from  (6)  and  the  elements  Ark. 

Equations  analogous  to  the  above  can  easily  be  written  for  the  case  that  (A)  differs  from  (a)  in  the 
elements  of  a  given  row  rather  than  a  given  column. 

1  The  Texas  Co.,  Beacon,  X.  Y. 

'  Abstracted  from  an  article  by  Jack  Sherman  and  Winifred  J.  Morrison,  The  Annals  of  Mathematical  Statistics,  >1  (March  1950). 

'  Presented  at  the  Twelfth  Summer  Meeting  of  the  Institute  of  Mathematical  Statistics,  Boulder,  Colo.,  1949. 
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Computation  of  an  Inverse  Matrix  by  Means  of  IBM  Machines 

The  foregoing  relationship  for  obtaining  the  inverse  of  a  matrix  differing  from  a  given  matrix  only 
in  the  elements  of  a  given  column  can  be  utilized  to  provide  a  convenient  means  of  computing  an  inverse 
matrix. 

Starting  with  a  given  matrix  (a)  and  its  inverse  (6),  the  inverse  (/?)  of  any  other  matrix  (a)  may  be 
obtained  by  N  applications  of  the  foregoing  procedure  for  replacing  the  elements  of  a  given  column. 
In  other  words,  (a)  may  be  transformed  into  (a)  by  N  steps,  the  first  involving  the  replacement  of  the 
elements  of  the  first  column  of  (a)  by  the  elements  of  the  first  column  of  (a),  the  second  step  involving 
the  replacement  of  the  elements  of  the  second  column  of  (a)  by  those  in  the  second  column  of  (a),  etc. 
Corresponding  to  each  of  these  changes  the  inverse  may  be  computed  according  to  the  foregoing  methods. 

If  (a)  and  (6)  are  taken  to  be  the  unit  matrices,  then  the  method  described  of  systematically  replacing 
the  column  of  the  unit  matrix  by  those  of  (a)  is  exactly  equivalent  to  the  familiar  method  of  S3'stematic 
elimination.  However,  if  the  matrLx  (a)  to  be  inverted  is  similar  to  a  matrix  (a)  that  has  previously 
been  inverted,  then  better  control  of  the  computations  can  be  achieved  by  using  the  presently  described 
method. 

The  method  of  obtaining  an  inverse  by  the  procedure  described  in  this  paper  is  being  carried  out  on 
IBM  machines  at  the  present  time  in  connection  with  spectrometric  analyses.  It  has  been  found  to  be 
more  efficient  than  the  orthodox  methods  of  systematic  elimination.  Furthermore,  this  method  provides 
considerable  flexibility,  and  obvious  saving  of  computations  in  cases  where  (a)  differs  from  (a)  only  in 
a  few  of  the  A^  columns. 

If  matrix  (a)  is  transformed  by  deleting  a  row  and  column  or  by  augmenting  a  row  and  column,  the 
corresponding  inverses  of  orders  iV— 1  and  A^+1,  respectively,  can  be  easily  obtained  from  (6).  In  the 
case  of  a  symmetrical  matrix,  it  is  possible  to  obtain  simple  relationships  for  computing  (/3)  from  (6)  for 
the  case  that  (a)  differs  from  (a)  in  the  elements  of  the  kih  row  and  column. 
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19.  Results  of  Recent  Experiments  in  the  Analysis  of 

Periods  Carried  Out  in  the  Istituto  Nazionale 

per  le  Applicazioni  del  Calcolo ' 

Galtano  Fichera^ 

hetjit)  be  a  function  that  has  empirically  given  values,  real  or  complex  in  the  interval  — 1<^<  +1. 
.  The  analj^sis  of  the  periods  of  the  function /(O  means  the  following:  Evaluation  of  the  first  natural  num- 
i  ber  n  and  the  constants  71,  72,  .  .  .,  7n;  ^1,  ^2,  •  •  •,  ^n,  such  that 

m-i^ijce'''  <€,  (1) 

where  e  is  a  known  positive  constant. 

A  method  of  the  analysis  of  the  periods  will  be  outlined  briefly.  AsJ(t)  is  given  empirically,  it  is 
not  admissible  to  use  expressions  that  involve  differential  quotients  of  f(t),  but  definite  integrals  of  f(t) 
or  products  oiJ{t)  with  other  known  functions  may  be  used.  The  number  n  must  not  be  too  large.  The 
method  is  the  following.     If  we  had  exactly 

k  =  l 

then  J{t)  would  represent  a  solution  of  a  differential  equation  with  constant  coefficients 

i  i:c,f'Kt)=o,  c„=i.  (2) 

k  =  0 

If  <Psit),  (s=  1,2,3,  .  .  .)  represents  a  complete  system  of  functions  in  the  interval  [—1,4-1]  in  the  sense  of 
Hilbert,  then  (2)  is  equivalent  to 

'  r  <Ps{t)f:CJ''\t)dt=:0,      8=1,2,3,..., 

J-l  fc  =  0 

which,  after  a  number  of  partial  integrations,  can  be  written  as 

1    z;s^."-^'(i)^.+x;w'-''(i)«*+2(-irc,  f  <p:''\t)Mdt={-ir-'  f  <p:"'{t)f{t)dt,  ..=1,2,3, . . .,  (3) 

t=l  i  =  l  A-  =  0  J-l  ,/-l 

I  with 

/3.=(-iy-'±;cj''^-"(i),  a,={-iy±c,f''-H-i). 

;  k=i  k^i 

But,  asy(0  is  only  given  empirically,  the  integration  in  (3)  can  be  performed,  but  like  the  ai  and  /ij  cannot 
I  be  evaluated  as  they  involve  difl"erential  quotients  of /(O  at  t=±\.     Therefore,  one  considers  the  «< 

and  /3i  in  (3)  as  new  unknown  parameters.     (3)  is  then  a  system  of  an  infinite  number  of  lin(>ar  equations 

.  with  'in  unknown  parameters  at,  fit,  (i=l,2,  .  .  .  n)  and  Ck  {k=Q,\,  .  .  .  n—\).     The  matrix  of  the 

I  coefficients  of  the  system  (3)  has  the  rank  3n,  if,  for  instance,  (ps(t)=e^-^  and  the  sequence  of  the  X,  has  the 

property 

Max  lim  X,=  + oo,     Mia  iim  X,=  —  c»  .  (4) 


'  This  is  a  report  on  an  address  delivered  by  M.  Ticone  and  F.  Conforto  at  a  incetinR  of  the  ().  A.  M.  M.  at  Freilnirn  i.  Dr.,  March  lOSI. 
'  University  of  Trieste  and  Italian  Institute  for  the  Application  of  Calculus. 

125 


Provided,  that/(i)  does  not  satisfy  a  linear  homogeneous  differential  equation  with  constant  coefficients 
of  the  order  m<jn,  the  sequence  X,  can  be,  for  instance,  the  sequence  0,  —1,  +1,  —2,  +2,  —3,.  .  .  . 
In  other  words,  one  considers  for  each  empirically  given /(O  the  system  (3)  with  <ps(f)=e^'^,  where  (4) 
is  valid  for  the  X,. 

Numerous  numerical  experiments  that  have  been  carried  out  in  the  Istituto  Nazionale  per  le 
Applicazioni  del  Calcolo  have  shown  that  the  system  (3)  for  a  number  of  p  equations  gives  solutions,  that 
tend  to  the  exact  values  of  the  Ck  with  increasing  p. 

If,  therefore, /(O  is  given  empirically  and  the  C*  (^=0,1,2,  .  .  .)  are  unknown,  then  it  is  indicated 
to  solve  the  system  of  p  equations  of  the  system  (3)  by  the  method  of  least  squares  for  n=l,2,.  .  .  . 
If  n  is  the  first  number,  such  that  for  increasing  p  the  Ck  converge  to  Ck,  then  it  can  be  conjectured  that 
f(t)  approximates  an  integral  of  (2),  formed  with  these  coefficients  Ck-  The  roots  2*  of  the  characteristic 
equation  of  this  differential  equation  can  be  determined  then  and  the  jk  are  solutions  of  the  system  of 
p  equations 


J-l  k  =  l         J-1 


This  system  can  be  solved  by  the  method  of  least  squares,  provided  that  for  increasing  p  the  solution 
tends  to  a  limit  jk.  If  the  7*  and  Zk  are  evaluated,  one  can  check  if  (1)  is  satisfied.  In  case  it  is  satisfied, 
the  problem  can  be  considered  as  solved,  otherwise,  one  has  to  perform  the  same  procedure 
with  71+1,  n-\-2,.  ...     In  case  n  is  too  large,  the  problem  has  to  be  considered  as  unsolvable. 
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